Uncounted: The hidden death toll of the COVID-19 pandemic

2022 Shortlist

Uncounted: The hidden death toll of the COVID-19 pandemic

Country/area: United States

Organisation: USA TODAY network

Organisation size: Big

Publication date: 23/12/2021

Credit: Dillon Bergin, Betsy Ladyzhets, Jake Kincaid and Derek Kravitz of the Documenting COVID-19 project; Sarah Haselhorst, The Clarion-Ledger; Ashley White and Andrew Capps, The Daily Advertiser; Rudi Keller, The Missouri Independent; Nada Hassanein, USA TODAY

Biography:

Since the beginning of the pandemic, the Documenting COVID-19 project at Columbia University’s Brown Institute for Media Innovation and MuckRock have worked to figure out how public health records and resulting data influences and shapes government policy. Death certificates are among the most influential records.

For this story, journalists from five newsrooms reviewed CDC mortality data at the county level. They compared those figures with models developed by the CDC and a team of demographers at Boston University, collected death certificates and documents and interviewed more than 100 medical examiners, coroners, public health experts, families and policymakers. The team at Boston University worked with the journalists on this project, providing models of expected deaths in every U.S. county and identifying areas of potential underreporting of COVID-19 deaths.

Project description:

For our Uncounted investigation, journalists from five newsrooms worked together to analyze CDC mortality data and follow that data to where it originates at the local level, through death certificates. We compared official COVID death figures with models developed by the CDC, in coordination with a team of demographers at Boston University; we collected death certificates and other primary source documents and then had medical examiners and physicians review them for errors and omissions; and we interviewed more than 100 medical examiners, coroners, public health experts, families and policymakers

Impact reached:

What we found: Short-staffed, undertrained and overworked coroners and medical examiners were all but unified in when and how to investigate a possible death from COVID-19. Some took the family’s word for what they believed was their loved one’s cause of death. Others didn’t review medical histories or order tests to look for COVID-19, but expected the state or family members to provide documentation. Death investigators and some physicians attributed deaths to inaccurate and nonspecific causes that are meaningless to pathologists, but closely resemble symptoms of COVID-19.

Our investigation reveals the country’s central problem with tracking COVID-19 deaths: Where people live and die has a lot to do with the accuracy of their death certificate. Some deaths are investigated with state-of-the-art technology and expertise; others don’t go beyond a phone call with the family.

Much of the impact from this project, published in three parts in mid-to-late December, is still evolving. But the CDC said in a statement that our findings go beyond what they are able to provide, and said forthcoming working papers would seek to address some of the more common reporting errors.

“We’re trying to push out as much information as we can, but we don’t have the resources to go digging in all of these counties. So it’s great that you’re doing this,” Bob Anderson, the CDC’s chief of mortality statistics, told us. “The sort of information that you’re digging up can help us, potentially, to improve the quality of the data.”

Techniques/technologies used:

The team at Boston University worked with the journalists on this project to provide them with models of expected deaths in every U.S. county, a level of granularity never before reported on a national scale. The team then identified states and counties that had highest rates of deaths that were both more than any normal, pre-pandemic year, but weren’t attributed to COVID.

This modeling data was created using 10 years worth of CDC mortality data and a regression analysis. We supplemented this data with our own point-in-time analysis of 2020 and 2021 CDC mortality data, from its newly-released WONDER API.

In the areas flagged by models, reporters found an unusual increase in deaths from natural causes, especially at home, and spoke with local corners and medical examiners who would likely be investigating and certifying the death certificates in these cases.

We’ve made the data publicly available as a repo and embeddable Google Sheet, along with a searchable data viz by county:

What was the hardest part of this project?

Mortality data, at the local, state and federal levels, only tells part of the story.

As our series found, many of the errors and omissions on death certificates reflect a long-simmering problem of conflicting or unclear CDC guidance, a lack of training and resources or general apathy or politicization of the pandemic. As a result, we struggled to reconcile these data discrepancies and had to rely on on-the-ground reporting to shore up the differences.

Many of the answers we received about the number of unexplained excess deaths in a particular county, specifically from Louisiana, Missouri and Mississippi, were revealing and not surprising. But they had not been fully explained before and elected coroners have yet to be held accountable for these errors and the incompleteness of their data.

We plan to continue this work throughout 2022, in an attempt to expand on these findings.

What can others learn from this project?

As part of the project, Documenting COVID-19 is sharing the data used for the larger investigation to help local newsrooms investigate how COVID deaths are certified and counted in their community. We’ll be updating this spreadsheet with new data and tools for reporters in the coming months. We also held a webinar on our reporting process and shared our slides here. We’ve received tips through our callout form from New York to Wyoming, and are already working with reporters in the USA TODAY Network in Georgia, Massachusetts, Wisconsin and five other states to replicate these stories locally.