Tracking Vaccinations and a Broken Public Health System

2022

Tracking Vaccinations and a Broken Public Health System

Country/area: United States

Organisation: The New York Times

Organisation size: Big

Cover letter:

We are proud to submit “Tracking Vaccinations and a Broken Public Health System” for a 2022 Sigma Award for data journalism.

While the vaccine supply remained low, The New York Times tracked daily in every state the inconsistent and ever-changing rules to determine who was eligible for a vaccine, a target not undertaken by any other publication.

This data collection provided a backbone for hundreds of stories. The reporting showed that a person could be eligible for the vaccine in one county, but not in the next, as well as racial and political disparities. The Times also studied how prepared the United States was for the next pandemic. And journalists showed that vaccine mandates appeared to have little impact on overall vaccination rates.

The Times’s effort to collect coronavirus vaccination and public health data and translate it into stories and public service trackers has been geared toward helping readers find and navigate the ever-changing, inconsistent and often confusing rules of the vaccine rollout. And it has focused on using this data, bolstered by traditional shoe leather reporting, to show readers the inequity of the vaccine distribution system — in the United States and around the world. Even as the vaccine has become more widely available, this data has helped show how ill-equipped the public health system is to handle a future pandemic, not to mention the ongoing surges the country continues to face.

The vaccination eligibility tracker attracted enormous attention from readers. As The Times added age and job categories to its tracker, states began expanding eligibility rules — faster than they originally planned — to include those categories, perhaps not to be outdone by other states. As The Times added categories, reporters from other news organizations began asking about disparities in those categories at White House and state health department press conferences.

The Times’s vaccination data tracking and analysis have been cited by numerous other news organizations — including Forbes, National Public Radio, The Rachel Maddow Show, Mother Jones and The Guardian — as well as by academic researchers, the White House, local government agencies, the Congressional Research Service and consulting groups, like McKinsey. The Times’s analysis of public health department surveys was cited in a letter from the National Association of County and City Health Officials to Attorney General Merrick Garland, requesting federal protection for local public health workers, and the investigation was cited by many other news organizations and featured on “The Daily” podcast.

Description of portfolio:

Beginning in December 2020, Times reporters conducted multiple rolling surveys of state and local governments to track basic vaccination data. Once the C.D.C. began reporting daily doses, Times journalists shifted their focus to other areas where gaps existed.

On a daily basis for months, journalists scoured government websites and contacted state and local government agencies when the rules were not clear. Quickly, it became clear that the eligibility rules were prioritizing people in unequal ways as the vaccine supply remained low. Times journalists began digging into that prioritization and how it differed from place to place.

Once all adults were made eligible for the vaccine in every state, The Times began an effort to study the people who chose not to get shots and to look closely at the public health system.

The Times built on the process it developed last year for collecting and vetting Covid-19 cases and deaths as it waded into the even murkier territory of recording vaccinations, demographics, hesitancy, eligibility rules, vaccine mandate rules and public health survey results. It created several internal databases dedicated to tracking this data. Each database had a strict methodology for entry and verification, developed by Times journalists. The Times mainly used R, Microsoft Excel and Google spreadsheets to analyze data.

The stories in this series used underlying data gathered from extensive surveys of state and local health departments.

For example, for “Why Public Health Faces a Crisis Across the U.S.,” Journalists sent an extensive survey to every county health department in the U.S. (about 3,000 departments) and they submitted more than 100 public records requests. The team analyzed state legislation related to the coronavirus that had been introduced in every state (about 8,000 bills, in total). The team watched more than 100 hours of video of public meetings. Danielle Ivory and Mike Baker interviewed more than 150 local health officials, lawmakers and public health experts.

The reporting showed that local health officials were actually less equipped to confront a pandemic than they were at the beginning of 2020.

For “Least Vaccinated U.S. Counties Have Something in Common: Trump Voters,” using federal health data, Times journalists analyzed vaccine hesitancy estimates for every county and state. Using R, Times journalists used a regression analysis to determine how strong the relationship was between a county’s voting record and its vaccination rate. Joining real vaccination dose data from the C.D.C. with data from the U.S. Census Bureau, journalists found that, even after accounting for a variety of factors, the political divide still remained.

It was important to create strict methodology and definitions, so that journalists would interpret information in a standardized way. This was particularly important when updating the eligibility tracker because the rules were often open to multiple interpretations.

The vetting system ensured that information about vaccine doses, eligibility, demographics and other vaccination-related records were sourced properly and checked by multiple people before publication. For example, a journalist updating eligibility rules associated with a state was required to save a copy of written confirmation of the rules as a PDF and add it to an internal database of source documents and link to that PDF in the eligibility database. That journalist would also check the largest counties in that state and do a scan for small counties with differing rules — and that sourcing would be saved to the source database. That journalist would then flag the state for a fact-check from at least one additional journalist. Every state in the eligibility database was checked in this manner every weekday.