Prize committee’s comments:

This entry turned an in-depth, technical investigation into how TikTok’s algorithm shows or hides videos about the war against Ukraine, based on where people live, into a beautiful, organic, and captivating presentation. Despite going into it expecting the result (that young men in Russia do not see TikTok videos about the war), seeing the side-by-side comparison of what TikTok showed to different users based on geography was striking and emotionally resonate.

Project description:

When Russia went to war against Ukraine and the media withdrew from the area, the dance app TikTok suddenly became an important channel for news from the war.
But did TikTok show the war to everyone? With the help of innovative and advanced programming, artificial intelligence, local proxies and a good dose of creativity, NRK was able to send TikTok robots to the war zone to document what the Chinese company showed young men in the neighboring cities of Belgorod (Russia) and Kharkiv (Ukraine).
Finally we could reveal that TikTok did not show the war to its Russian users.

Impact reached:

The article has around 360,000 readers in Norway. It also went viral and was translated by various users all over the world. Therefore, we also made an English version to ensure that the translation was correct. This meant that we reached a new 85,000 readers and received, among other things, a recommendation in the New York Times newsletter.
The fact that the case received so much attention abroad also led to several people contacting us to learn more about how we worked or to offer cooperation. We have held a number of lectures for data journalists in Norway, Sweden and Germany. We have also shared our methods in an interview with the Global Investigative Journalism Network (GIJN).
Since publication, TikTok has quietly started showing new videos from both Russian and international users – completely contrary to what has been the company’s stance externally.

Techniques/technologies used:

First of all, we used “residential proxies” to make the TikTok algorithms believe that our traffic came from the Russian border city of Belgorod and the Ukrainian border city of Kharkiv.
The next step was to build a TikTok bot with the Python programming language that could scroll through thousands of records and extract information. This was very challenging, since TikTok has strict monitoring of all activity, and kicks out users who engage in bot-like behavior.
It quickly became clear that the Ukrainian bot got to see videos from the war from the start, while the Russian one didn’t see a single tank. We therefore tried to “help” the Russian bot to see more of the war.
TikTok’s algorithm works in such a way that you train it to serve more of what you like. We therefore collected subject tags and keywords that were associated with the war. In addition, we used Microsoft’s tool Azure Computer Vision, which is an artificial intelligence that can look at images and provide a description of what is in the image. We fed this AI words like guns, tanks, the military, soldiers, explosions and so on. Then we programmed the bot to look longer at videos that contained the topic tags and keywords we had collected or images that the AI recognized.
Yet the war never appeared in Russia. We then tried via our proxy in Russia to view the videos of acts of war that the Ukrainian bot had seen, but it was not possible. So we could state that TikTok does not show the war to its Russian users.

Context about the project:

Ethical considerations

Use of videos from TikTok
We wanted to show readers a selection of the actual videos TikTok served to the Russian and Ukrainian user. Especially the very first videos the bots saw, because already there we saw a huge difference. We believe we can use the news right to show the videos, because it is very important to make people understand how the algorithms help to give people in Russia and Ukraine a different picture of the war. The videos were also publicly available to everyone. However, it has been important to carefully go through each video to be shown, and assess how much of, among other things, faces, text and recognizable features/clothing that we can show. For example, one of the first videos shows a child donning military gear. Here we have covered the face, and we have also done this in many of the other videos. We have gone carefully 9 through all vid

Fake users
We violate TikTok’s guidelines when we create fake users and create software that automatically scrolls through TikTok. We did it anyway, because we believe it is important that someone checks and shows how the algorithms work. TikTok is also an important source of news for young people in particular – in our case young men who have to decide whether to participate in a war. It is critical for democracy and society that citizens understand how the algorithms determine what content you get in your feed.

Residential proxies
Residential proxies make it possible to “trick” a website or service into thinking you are somewhere else in the world than you really are. It is more difficult to see through residential proxies than, for example, VPN solutions, because the internet traffic goes through the IP address of a regular internet user. However, this also creates ethical challenges. An internet user can potentially share internet traffic with others without knowing it. It would have been very unfortunate if NRK contributed to something like this. Therefore, it was important for us to use one of the largest providers of such services, even if this also meant one of the most expensive.

What can other journalists learn from this project?

How to tell a data story?
It is nice to have a lot of data, but in order for many people to take it in, you must also be able to communicate it in a good way. That’s why we thought about the storytelling right from the start. The reason why we sent the bots to Belgorod and Kharkiv was that we had navigated the Russian-Ukrainian border in Google Maps to find war-affected cities in Ukraine that had a Russian city just over the border. This geographical proximity (only 8 miles between the towns) gave a stronger nerve than if we had covered the whole country. We also chose to let our own work process play an important role in the matter. That way the reader could join in and see how we built the bots, what kind of properties we gave them, how things changed when we changed the settings and how different videos the bots Alexei and Nykolai got.
It was also a point that the reader should feel that he or she was in the TikTok universe. That’s why we chose a presentation template where you scroll down one card at a time – just like in the TikTok app.
Project link 2 is a link to a more detailed report on our working methods. It is in Norwegian, but can be translated with Google Translate.

Project links:

https://www.nrk.no/osloogviken/xl/tiktok-doesn_t-show-the-war-in-ukraine-to-russian-users-1.15921522

https://www.skup.no/sites/default/files/metoderapport/2022-11/nrk_ug_metoderapport_dataskup.pdf