ProPublica placed a bet at the beginning of the pandemic: that by creating our own databases of local court cases from cities and states across the country, we could find unique and impactful stories about how the pandemic was affecting people who are on the margins of poverty in the United States. The result was a series of investigations into how debt collectors and landlords used courts to squeeze the working class despite the toll of the pandemic on their health and finances.
Additional publication dates:
By shining a quantitative light on the punitive methods used by landlords and debt collectors to extract money from the poor during the pandemic, our reporting triggered local and national changes.
In response to our reporting about evictions filed in violation of the CARES Act, landlords dismissed more than 100 eviction cases that they had illegally filed against tenants. While reporting this story, it became clear that many tenants were confused as to whether their property was covered under the new law, so we built an app to help renters determine if they were covered by the federal ban. The app was cited by Senator John Tester during the Senate Committee on Banking, Housing, and Urban Affairs on June 9, 2020.
Our joint investigations team with the Texas Tribune used court data in Texas to uncover a lender who filed thousands of lawsuits against low-income borrowers during the pandemic. When the company learned about our investigation, it dismissed all its pending lawsuits across the country and temporarily suspended new suits. The company would not confirm the exact number of cases it dropped, but our data shows it filed about 10,000 lawsuits in Texas alone during the first half of 2020.
Another investigation found that one Indiana school district filed nearly 300 lawsuits against parents during the pandemic, most for unpaid textbook fees. After contacting the district, it announced it would forgive the unpaid fees of one family mentioned prominently in the story and vacated the small claims judgment filed against her.
Citing our reporting on debt collection cases filed or pursued by debt collectors and banks during the pandemic, Senators Elizabeth Warren and Sherrod Brown sent letters to Capital One and other companies named in our story, demanding they suspend aggressive collection activities during the pandemic.
We scraped court cases from more than a dozen of the nation’s largest counties and some statewide court systems to produce our analyses for this series. That entailed writing scrapers tailored to each site in Python and Ruby, and collating the data across jurisdictions into a single sensible format for our reporting. We also geocoded and linked data sets (including court cases, lists of federally-backed properties and parcel map files) together using PostGIS, Mapbox and Vue.js to analyze illegal eviction filings.
Our database resulted in an interactive news app where readers could look up whether they were covered by the CARES Act and state eviction moratoria, as well as to find properties with large numbers of eviction filings during the pandemic. We used Postgres to store the data and ran queries in SQL and Pandas to analyze the data for various stories. Plaintiff names were extensively cleaned using regular expressions and text clustering in OpenRefine.
What was the hardest part of this project?
It was a significant technical challenge to coordinate scrapers made by multiple people for multiple court websites to run weekly and create an accurate and up-to-date picture of debt and eviction cases filed in counties across the country. We made a suite of in-house tools and processes for doing so in a uniform fashion. Still, each had to be hand-crafted and vetted for completeness and accuracy, and entailed speaking with legal and housing experts on the ground across the country to understand each unique court system and local context.
Additionally, our decision to link data together to analyze evictions at the building and ownership level gave us a unique tool — one that, to our knowledge, no other organization has — that allowed us to publish unique insights on eviction practices during the initial waves of the pandemic.
What can others learn from this project?
If a database doesn’t exist that you want, create it yourself. Likely, that means you’ll have new insights that reporters, researchers and government agencies haven’t uncovered before. Additionally, if there is information that isn’t readily available to the public, or is siloed across different agencies, building tools to make it accessible for everyone is not just a public service, but can inform concerned citizens and ultimately affect policies.