2022
YouTube offered conspiracy theories and religious content to Finns searching for Covid vaccine info—Yle’s study exposes the social media giant’s blindspot
Country/area: Finland
Organisation: Yle (Finnish Broadcasting Company)
Organisation size: Big
Publication date: 30/05/2021
Credit: Satu Helin, Juha Rissanen, Päivi Koskinen
Biography:
Satu Helin is a Data Journalist and foreign news reporter at Yle World news desk. She studied data journalism at Columbia Journalism School with the scholarship by Helsingin Sanomat foundation in 2019-2020.
Juha Rissanen is a Graphic Designer at Yle Data desk
Päivi Koskinen is a producer at Yle World news desk. Previously she has been working as a foreign news reporter.
Project description:
After Covid pandemia hit the world, YouTube had promised to strongly limit the spread of the vaccine disinformation and conspiracy theories on its platform. However, when people searched for vaccine information on YouTube in Finnish, the platform was not able to keep its promises, Yle’s investigation revealed.
Instead of official and medical videos that YouTube had promised to raise, people were shown religious and covid conspiracy videos when they simply searched with the word “koronarokote” (covid vaccine in Finnish). In English, the results were opposite.
YouTube corrected the problem after seeing the results of Yle’s investigation.
Impact reached:
YouTube’s owner Google admitted that YouTube’s misinformation filter had not been active when users typed in the Finnish compound word for coronavirus vaccine into the search field.
The platform corrected its filters after we contacted it with the outcome of our analysis in late May.
That radically changed the search results millions of Finnish YouTube users got when they looked for videos with the word “covid vaccine”.
Instead of religious and anti-vaxx videos, they got medical, official and news videos among their top 10 results after Yle’s contact with Google.
In Finland, under 35 year-olds started to get vaccinations in June-July, and under 25 year-olds in July-August. This means the age groups, who according to the specialist, use Youtube as their prior search engine, got more truthful and reliable information before they had to make a decision on covid vaccination.
According to Google’s own data, the word ‘covid vaccine’ was one of the most popular pandemia-related search terms in FInland in 2020 and 2021.
Without this story, the dis- and misinformation would have continued to spread freely in Finnish YouTube and Finnish users might still be affected by it.
The story raised discussion on the big tech carelessness when it comes to small languages. We also received requests for help in repeating the investigation in other languages.
In Finland, the story was picked up by several news media. Internationally, the translation got coverage by newsletters such as AlgorithmWatch and Coda. It was also spread by the most prominent names of the big tech journalists.
Techniques/technologies used:
In February 2021, we created eleven different YouTube accounts, including two control ones.
The accounts modelled YouTube’s most popular intended uses, which we gleaned from the platform’s own data, available research as well as the video giant’s most popular channels.
We wanted to see if people watching different content got different search results on covid.
The accounts were created on the same day using a location-masking VPN service. The VPN server was located in Finland. In terms of user information, we only provided YouTube with a birth year, a requirement for using the service. We identified Finland as our location.
We created a 50-70 video playlist for every account.
Each account watched its own playlist. For example, the account consuming news watched different news videos and American current affairs programmes. An account focused on entertainment viewed music videos and comedy content. All accounts saw content in both English and Finnish.
After watching their playlists, each account searched YouTube using the most common covid related words defined by Google Trends.
We saved the first 100 search results for each account in Goolge Sheets. Our searches resulted in 4,800 videos in Finnish and 3,598 in English from 477 channels.
The working group then watched hundreds of YouTube videos to classify their content, and ended up defining 13 different categories such as mainstream media, government agencies, religious entities.The results were analysed with Python Pandas.
YouTube search results did not appear linked to whether a user watched news or make-up tutorials. YouTube’s search algorithm simply did not block questionable content on covid from any account if a user did a keyword search.
Yle asked YouTube why government messaging on coronavirus vaccines ranks below religious and suspicious vaccine content on the Finnish-language site.
Just two days later, YouTube’s search results dramatically shifted.
What was the hardest part of this project?
– Creating a valid hypothesis followed by a solid investigation plan that was restricted enough. Without that everything else is useless.
What can others learn from this project?
1) The results one gets when studying social media algorithms in action may vary profoundly depending the language used. The algorithm investigation is heavily dependent on English findings, but there is an urgent need for investigation on how the algorithms work and affect the societies in smaller languages.
2) Our study could be repeated in other languages.
