The Telegraph’s top-performing coverage during the 2019 UK General Election all had data journalism at the heart of it. Our interactive results forecaster was more accurate than the official exit poll, and our tactical voting tool, the only one by a major publisher to feature a guide for both Remain and Leave supporters, was our second most prolific subscription-driving piece of content ever. The data team’s decision to make a live Brexit-focused graphic one of the centrepieces of our results page paid off with record audience and engagement figures.
The Telegraph’s live 2019 General Election results page had the reader at the heart of it. We were the only UK publication to have a Brexit-focused graphic live-updating on our results page, with our histogram of Brexit vote versus party gains proving essential in telling anyone the story. Brexit proved the driver of the swings seen in the election, and our gamble with making a live Brexit-focused graphic one of our centrepieces paid off. It meant that we were able to provide visual-led updates of the stories of the election (e.g. https://twitter.com/Ashley_J_Kirk/status/1205325253800529927) without having to due any heavy-lifting in ggplot or Illustrator. It also helped with a high reader engagement time, which in turn meant that the page led Google more than any other news publisher and attracted record page view figures from our core subscriber audience
Our data-driven stories for the election were among the most-subscribed pieces for The Telegraph this year. When it came to the results analysis, our Data Journalism team of three was aided due to the fact that we had an innovative toolkit of live-updating, diverse and story-driven results graphics. These graphics went beyond the standard live bar chart or map which show seat totals; we took a risk on understanding what the stories of the night were likely to be, as well as audience’s needs, and created the graphics based on this.
Our results page contained nine live-updating graphics, and each of these was a component that could be embedded in any other article. These individual embeds – called Particles – were then embedded in lots of pieces of online content, which helped lift the bar of our overall visual storytelling. We had millions of pageviews and our data journalism included some of the Telegraph’s highest-ever subscribed content to date.
There are vast arrays of technologies used to produce the live election data. Starting at the beginning with PA’s FTP server we ping it at regular intervals to keep track of the XML data files there are to process. If there are new files, we download, convert to JSON and then process in one of our 6 serverless AWS lambda functions we have for the general elections data. Keeping track of all these data files is achieved by storing the file name and modified date into a cache, we use Redis. The processed JSON files are also stored on an AWS S3 bucket for future reference.
All the data we process from PA automatically populates a programmatically generated Google Spreadsheet that allows our Data Journalism team to analyze and pluck out data that can drive live analysis.
We then download the data from the same Google Spreadsheet using another serverless function, which this time will have the Telegraph’s picks and comments added. Using this data we generate JSON data files, upload them to our public facing server and then invalidate our CDN cache to make sure the public see that latest results as soon as we upload them. The results page was created using our internally developed “Atoms” editor (a google doc’s style collaborative web page creator) in conjunction with technologies such as React to build each component on the page individually.
The whole tech stack was created using, AWS lambda function built in Node JS, AWS elasticache (redis), AWS Simple Storage Solution, AWS CloudFront, Adobe AEM, React js, Google Spreadsheets, Slack (for internal notifications of result and service status). The process from PA uploading new data to us then processing and displaying the information never took longer than 15 seconds.
What was the hardest part of this project?
The hardest part of this project was balancing the needs of numeric automation and human-generated analysis. Our overnight team had a lot of other work to be getting on with, such as a forecaster and a score of analysis pieces, and so it would ideally be the case that the results page powered itself. At the same time, we needed this page to read as a story in its own right, as our readers don’t just want to see a list of graphics without any explanation. The answer here came from old-fashioned hard work and collaboration with design and development teams, to ensure journalists could report on and analyse the results in a bespoke CMS that updated graphics in real time with zero production delay. Although a large organisation, the Data Journalism team at The Telegraph consisted of two full time journalists and one fixed-term contractor. It’s hard to staff up during a 24-hour results cycle and meet such extremely ambitious targets, and so our accurate and innovative coverage relied upon results graphics automation that still had stories at the heart of them.
What can others learn from this project?
In our opinion, election results pages shouldn’t be a data dashboard. While our data journalists, and the data journalism community as a whole, are very excited about all the numbers of election day, there is a difference between this interest and the interest of our readership. They want to know who won, why and what happens next – all when usually reading on a mobile phone while sleep-deprived.
A results page should be built to answer these questions in the most simple, comprehensible way, and our performance metrics prove that focusing on these more narrative-drive visualisations will lead to success. Graphics such as our Brexit histogram, line charts of key constituencies and a cartogram to show geographic distribution helped show this narrative. Each were used in multiple other stories aside from the results page as well, improving the impact of each individual component on the page.