2020 Shortlist
ADAM: The Automatic Data Article Machine
Category: Innovation (small and large newsrooms)
Country/area: Netherlands
Organisation: RTL Nieuws
Organisation size: Big
Publication date: 14/11/2019
Credit: Jasper Bunskoek, Wouter van Dijke, Rana Klein, Lara van Zuilen
Project description:
RTL Nieuws is a national news broadcaster. We bring stories that are engaging and relevant to our audience. To help us do this, we developed our own news robot, ADAM, the Automatic Data Article Machine. ADAM can generate many stories from a single dataset, for every city, neighbourhood, school etc. So far, we have used ADAM to write about traffic accidents and crime. ADAM generated 5.000 unique – and well-read – stories, customized for the 2.462 towns in The Netherlands. This allowed us to reach our audience on a local level and has already directly impacted local politics.
Impact reached:
The articles that we have created with ADAM so far, have been customized for the almost 2.500 towns in The Netherlands. Each time, within 24 hours after publication, we see that almost all of these articles have been read, from large cities to tiny villages. We see readers sharing the results that ADAM has generated for the place where they live and deeply engaging with this information. Readers are seeing confirmation of their own observations and even use this to influence local politics. Council questions have been submitted in several municipalities referring to an article that was written by ADAM.
ADAM has also been of use to local news outlets, who have shared our findings and created their own follow-up stories. By visiting dangerous crossroads we’ve found, trying to find explanations for an increase in crime or questioning their local government. While many local outlets do not have the capacity of hiring specialized data journalists, through ADAM we can create data stories on a local level. This way, we’re not competing with these outlets, but supplementing and aiding their work.
Techniques/technologies used:
Every story we publish is the result of a larger data investigation, which is usually done using Excel or Python. For the actual ADAM project, we developed a backend and user interface (UI) to enable journalists to publish large sets of data driven articles.
The UI enables journalists to upload datasets and to write templates for their stories. Data journalists use Python code within the text of their templates, to write conditions and make calculations. Pandas is used to manipulate the data within the UI. Furthermore they can use the UI to design and create unique infographics for there stories. Under the hood, we use Python to build a backend to process and analyse the data and to convert those into a collection of unique articles, including graphs, tables and images.
Both the backend and UI run in the cloud and are easily scalable. After the template is created, the text and required data visualisations for each story are generated and saved on an ElasticSearch server. Our API connects the generated content with our CMS, our website and our app to display these stories. For the end user, an ADAM story looks just like any other article we publish, including fonts, graphs, images and tables.
What was the hardest part of this project?
A strong prerequisite in this project was creating a tool that goes beyond simple fill-in-the-blanks templating. With ADAM, titles, graphs and entire paragraphs can be changed based on our source data. This ensures all generated articles are truly unique and relevant, while looking and feeling like any other human-written article.
In the end, the success of ADAM is defined by how useful it is to our audience. The simpler and more seamless ADAM seems to a reader, the better we can serve this goal. But this requires a lot of advance thinking, both during the data analysis and while writing templates.
We also wanted to create a reusable system that we can continually use to distribute the results of our data investigations.
A very important issue for us was figuring out the journalistic ethics of automated journalism. It is crucial to warrant the accuracy of every single generated story. To achieve this, it’s essential to us that journalists that use ADAM were involved in the development, so they understand how it works and what the limitations are. This way we can avoid a ‘black box’ and stay in complete control of the output. We’re also transparent in our use of ADAM, explaining what it is and how it works in every article written by it.
We have researched the use of AI technologies such as Natural Language Generation in ADAM, but even a nearly perfect algorithm is not good enough for this purpose.
What can others learn from this project?
We are not the first news outlet to experiment with automated journalism. We have been able to learn from what others have tried and hope to bring the development of this field another step forward.
The value of RTL Nieuws lies in a strong connection with our audience. ADAM shows how powerful it is to tell news stories that are close to the lives of readers and viewers. This can be a lesson for many journalists, especially with regard to data. Numbers can feel very abstract or complicated. The best way to make people care is by translating data to stories that hit close to home – literally, in the case of ADAM.
ADAM is a demonstration of how technology can help journalists with limited time and resources reach and serve large audiences.
We have received a lot of interest from other media in The Netherlands and abroad. We have given interviews in national newspapers and given presentations to other journalists, who are keen to learn from our experiences developing ADAM.
