Project description:

We revealed that a national shortage of GPs has left some surgeries with one permanent doctor caring for as many as 11,000 patients. 

The local scandals were uncovered using detailed analysis of every general practice workforce report released by NHS England between 2013 and 2019. From here we began our reporting, visiting the worst-hit (as well the best performing) surgeries to verify what the data had revealed and humanaise our findings. 

Looking into the demographics, we were also able to reveal that 1.3 million women in England do not have regular access to a female GP.

Impact reached:

Our investigation into the scale of the crisis in general practice services had a strong impact on our readership, as well as in the public debate. It was picked up by other national media such as BBC, Daily Mail, Sun, Telegraph, Metro.  

The project was published soon after the December general election, contributing to setting the new political agenda. By highlighting the most extreme cases where patients were suffering the most, this story renewed pressure on the Prime Minister to fulfill his electoral pledge to recruit 6,000 GPs by 2024-25.  

After the story was published, the Health Secretary Matt Hancock spoke out to reassure the public that the Tory pledge to tackle the NHS staffing crisis was underway.

The story received over 1,500 comments from online readers, showing a strong interest in the topic and high engagement. Since the findings were published, The Times has been contacted by hundreds of people sharing their personal experiences with their GP practices. Their stories have been gathered for a piece of community journalism, published a month later. 

The lookup interactive tool we created to allow readers to check how services have changed in their area, has been viewed over 23,000 times, with more than 3,000 searches.

Techniques/technologies used:

We analysed every general practice workforce quarterly report released by NHS England between 2013 and 2019. Data in the form of CSV files were imported and analysed in RStudio. For each surgery we filtered specific categories: number of patients, GP headcounts, full time equivalent GPs (by gender), registrars and locums. This allowed us to determine the number of patients per fully qualified GP in each surgery over time. From this, we listed the worst and best performing practices that needed further investigation. For each of them we cross-referenced NHS records and Care Quality Commission reports, which provided a better picture of the status of their care services.

We shared findings using Google Sheets, which gave us the opportunity to work on the data simultaneously.

Figures of appointment times, closed and dormant practices provided by the NHS for each Clinical Commissioning Group were also analysed in Google Sheets and then used to create interactive maps and charts to dress up the story online. For this, we used the web tool Datawrapper.

To personalise our story, we built a searchable tool that allowed readers to enter their postcode and check how GP services changed in their areas over time. 

We used the postcodies.io geolocation api to match up readers postcodes to their clinical commissioning group (CCG). The data for each CCG (including the latest figures for registered patients, doctors’ surgeries, GPs, closed and dormant practices and the number of patients per GP over time compared to the national average) was stored in separate JSON files. This meant we only had to load in the data for the relevant CCG and the interactive was a lot faster as a result. The interactive itself was built using React JS. The graphical element of the interactive was created using D3.

What was the hardest part of this project?

One of the most difficult parts of this project was to handle inconsistencies in NHS workforce data: this was due to a lack of clear guidelines on how surgeries should declare staff figures, but also to duplications in the case of practice mergers and closures. 

In order to be as complete and accurate as possible, we had to filter our results in RStudio, identify the best and worst practices manually and address them directly to confirm our figures. 

Once happy with the numbers, we spent quite some time discussing the best way to visualise them, so that readers could get key information for them. To overcome the issue of inconsistent figures for merged practices, we finally decided to use data grouped by clinical commissioning groups. 

Also, some last minute updates (on the day of publication) on the NHS website were easily handled by a simple re-run of our R script. 

It’s important to note that, for this project, we only relied on publicly available data that no one else had analysed in such great detail before: to figure out trends over time, to point out the best and worst performing practices, to look into demographics and identify a gender gap in surgeries, it took both meticulous digital analysis and in-depth reporting on the ground.

What can others learn from this project?

We can identify three key points we have learned from this project:

1. Using a programming language such as R to analyse data allows you to work with a clear script that can be easily tweaked and re-run for last minute changes in the data. 

2. Data led investigations are very useful to identify problems, but a combination with on the ground reporting remains very important. 

3. Data is a powerful tool to pinpoint strong news lines, as well as constructive angles for your story. 

Project links:

www.thetimes.co.uk/article/gp-crisis-nhs-shortages-mean-one-doctor-has-to-care-for-11-000-patients-v25nt2cdl

www.thetimes.co.uk/article/the-best-and-worst-hit-gp-surgeries-patients-queue-in-cold-to-see-a-doctor-td9fxmvmz

www.thetimes.co.uk/article/gp-crisis-third-world-conditions-at-surgeries-with-no-female-doctor-jgkqb8wcp