Bias Behind Bars

2021 Shortlist

Country/area: Canada

Organisation: The Globe and Mail

Organisation size: Big

Publication date: 24 Oct 2020

Credit: Tom Cardoso

Project description:

In late October, The Globe and Mail published Bias Behind Bars, an investigation into systemic racism in Canadian federal prisons.

Through a database of more than 50,000 inmates obtained via freedom of information request, we examined the structural biases inherent in the tools used to assess prisoners. Risk assessments are meant to be an impartial guide of who can be rehabilitated, and are steeped in decades of research. But, as The Globe discovered, these tools are fundamentally, powerfully biased against Indigenous and Black inmates, placing them in higher security classifications and assigning them worse odds of successfully re-entering society.

Impact reached:

The response to The Globe’s story was immediate. Within 48 hours, a member of the House of Commons’ public safety committee had pledged to conduct a parliamentary study of systemic racism in federal prison risk assessments. “This needs action,” he told The Globe. A day later, the study was officially announced with all-party support. Then, a day later, in a rare political acknowledgment of the challenges faced by Canada’s usually invisible prison population, Prime Minister Justin Trudeau himself said more had to be done by the federal government to fight systemic racism in prisons.

The investigative series’ impact has also extended beyond Parliament. Lawyers have used our findings at parole hearings, professors are using our stories as teaching material, and both senators and Canada’s correctional watchdog have referred to the series in recent statements. In early January, a civil rights lawyer filed a class-action lawsuit against the government on behalf of tens of thousands of inmates, arguing the use of these risk scores amounts to a deliberate discriminatory practice.

The strongest words, however, likely came from Jack Harris, the legislator who first proposed the parliamentary study. “It’s something that the government has to respond to,” he said. “The fact this has been done by somebody outside the system as opposed to the system itself is a condemnation of the effort that should’ve been done sooner and more effectively.”

Testimonials from those personally affected by correctional risk scores have driven home the importance of this form of data-driven investigative reporting. “Thank you for publishing a very important issue in The Globe and Mail newspaper,” a former inmate wrote. “I was held for an additional 10 years in high- and medium-security prisons by this biased manipulative system. … Thank you for telling our country the other side of the story; the truth.”

Techniques/technologies used:

The genesis of Bias Behind Bars was a freedom of information request, filed to the federal government, asking for years of data on federal inmates. Data-driven freedom of information requests of this sort are unusual in Canada – but after seven months of negotiation, the government eventually delivered a 750,000-row file documenting the lives of 50,000 people over a seven-year period.

We used a statistical programming language called R, combined with a data journalism analysis framework – startr, built and open-sourced by The Globe two years ago – to unpack and analyze the file.

After weeks of analysis, it became clear that simple descriptive statistics such as tallies, percentages and rates of change wouldn’t be enough to isolate the impact of race on someone’s risk assessments. Variables like age, gender, the severity of an inmate’s offence and their criminal history were all interconnected; combined, they led to an inmate’s score.

Instead, The Globe turned to statistical modelling. Over several months, and with the assistance of statisticians, criminologists and data scientists, we built a series of multivariate logistic regressions that controlled for all those variables and explained the impact of race on both men and women’s most important risk scores.

Finally, the insights gleaned from these models and the overall trends identified in the data were fed back into the reporting process, inflluencing the kinds of documents we sourced, the people we interviewed and the direction of the ultimate reported story. Those findings were also shared with The Globe’s design and graphics teams, who distilled the information into visual explanatory content, like the animated dots which guided readers through how inmates’ risk scores broke down by race.

What was the hardest part of this project?

For decades, federal watchdogs and prisoner advocacy groups have suspected race plays a role in determining an inmate’s risk scores. But, in report after report, analyses of risk levels only scratched the surface, looking simply at, for instance, the rate at which Indigenous inmates were classified to maximum security, or how frequently women were deemed to have a low potential for reintegration.

The Globe’s statistical investigation cut through this noise by doing something that had never been done: accounting for the myriad factors – age, offence severity, criminal history and so on – that play into a final score, effectively isolating the impact of race. The process of developing and fine-tuning a methodology for that analysis was by far the most complex part of the investigation.

It was also an analysis that likely wouldn’t have occurred without The Globe. During early conversations ahead of filing a freedom of information request for inmate data, several sources warned us the information we sought would never be released by Correctional Service Canada, the country’s federal prison agency. In fact, the agency itself had never undertaken an analysis of race and risk scores on the scale of what The Globe intended.

When the analysis revealed the biases in these critical prison tools, a second question emerged: How could the prison agency have missed the massive impact of race on inmates’ scores?

The answer, we suspected, was that they hadn’t.

In January, 2021, after months of reporting, we confirmed what had been an open secret within the agency for decades: An internal federal government document from 2004, obtained by The Globe, showed unequivocally that senior Correctional Service leadership had been warned of serious flaws in its security risk tool for 16 years – and yet the scale remains in use, unchanged, to this day.

What can others learn from this project?

Above all else, Bias Behind Bars reveals what’s possible when filing ambitious, data-driven freedom of information requests. Moonshots like these don’t always pan out, but when they do, they often lead to incredible stories. We’ve since begun filing more of these – even if they could take years to bear fruit.

As so often happens during large investigations, we pivoted several times early in the reporting process. Our original intent was to look at the diversity of Canadian juries – but that data didn’t exist. Instead, we began looking at sentencing patterns, but our freedom of information requests didn’t net enough data for that, either. Instead, they pointed to our ultimate topic: risk scores.

Experimentation was also essential to the project. This was the first time The Globe had built a statistical model for a story, and while I was already comfortable analyzing data, I’d never modelled a dataset. The process was intense and required a countless conversations with statisticians and academics, but resulted in a skill I’ll rely on in for future stories.

Finally, Bias Behind Bars shows the importance of using compelling characters and narrative journalism to pull readers through a story that would have otherwise been too dry, data- and policy-heavy.

While data-driven findings suffused all our reporting, those were carefully balanced against the story of Nick Nootchtai, an Indigenous man who spent his entire 12-year manslaughter sentence behind bars, the majority of it in maximum security. Instead of simply running readers through our findings and telling them how Mr. Nootchtai’s scores affected his prison life, we showed it, detailing the few prison jobs open to him, how his wife could never join him for conjugal visits and the fact that his high security score even denied him access to pencil crayons for arts and crafts.