Project description:

The New York Times used data to document the war in Ukraine, including potential war crimes.

In [Russia’s Attacks on Civilian Targets Have Obliterated Everyday Life in Ukraine](https://www.nytimes.com/interactive/2022/03/23/world/europe/ukraine-civilian-attacks.html) (March 23), journalists examined thousands of photos and videos, identifying for the first time more than 1,500 Ukrainian civilian buildings and other structures damaged by Russian attacks.

In [What Hundreds of Photos of Weapons Reveal About Russia’s Brutal War Strategy](https://www.nytimes.com/interactive/2022/06/19/world/europe/ukraine-munitions-war-crimes.html) (June 19), journalists collected hundreds of images, showing that 200 cluster weapons and 2,000 unguided weapons were found in Ukraine, often in civilian areas — the most comprehensive collection of such weapons.

Impact reached:

In March, 45 participating states of the Organization for Security and Cooperation in Europe, the world’s largest regional security-oriented intergovernmental organization with observer status at the United Nations, established an independent expert mission on violations committed in the war. [The report](https://www.osce.org/files/f/documents/f/a/515868.pdf), presented to the OSCE Permanent Council in April, cited the civilian targets story in documenting clear patterns of violations of international humanitarian law by the Russian Armed Forces in Ukraine. The authors wrote:

“It is even more evident, from reports of numerous official and unofficial sources, that tens of thousands of civilian objects have been damaged or destroyed in Ukraine, including houses, multi-storey residential buildings, administrative buildings, penitentiary institutions, police stations, medical and education facilities, water stations and electricity systems.” (Page 26 of [the report](https://www.osce.org/files/f/documents/f/a/515868.pdf).)

In July,[ a second OSCE report](https://www.osce.org/files/f/documents/3/e/522616.pdf) cited the weapons story as evidence that the invasion resulted in unnecessary, disproportionate harm to civilians in part due to indiscriminate attacks in densely populated areas with, among other munitions, cluster weapons. The authors said The Times “discovered photographic evidence of widespread use of cluster munitions in a variety of civilian settings. It was noted that the majority were unguided, with a proclivity to cause collateral damage to civilians. It also discovered instances of other types of weapons that may be illegal under international law, such as land mines.” (Page 49 of [the report](https://www.osce.org/files/f/documents/3/e/522616.pdf).)

In June, Evan Ellis, a professor at the U.S. Army War College, cited the civilian targets story in [testimony](https://docs.house.gov/meetings/FA/FA07/20220720/115002/HHRG-117-FA07-Wstate-EllisR-20220720.pdf) before the Congressional House Foreign Affairs Subcommittee as evidence of “brutality in Russia’s targeting of civilian populations in the Ukraine.”

[Wikipedia’s page](https://en.wikipedia.org/wiki/War_crimes_in_the_2022_Russian_invasion_of_Ukraine#cite_ref-46) outlining potential war crimes committed during the war cited the weapons story as evidence of Russia’s use of cluster munitions, unguided missiles and landmines.

Techniques/technologies used:

After Russia invaded Ukraine, software developers at The Times used Crowdtangle and other apps to scrape Twitter, Facebook and Telegram posts as well as press releases from Ukrainian government accounts and send them to a Slack channel. To help reporters search the posts, the developers created a custom translation bot that grabbed every message and instantly responded with the English translation by running each message through the Google Translate API.

For the civilian targets story, journalists used this bot, as well as The Times’s database of photos from wire journalists and its own photographers on the ground, to manually collect evidence of more than 1,500 attacks on Ukrainian civilian infrastructure in Google Sheets, including at least 23 hospitals and other health structures, 330 schools, 27 cultural buildings, 98 commercial or food-related buildings and 900 homes. The reporting showed how, in just weeks, normal everyday life for many people in Ukraine was obliterated. The images, all evidence of potential war crimes, were cross-referenced with government and N.G.O. records, Google Street View and other websites in both English and Russian, to help pinpoint locations.

For the weapons story, journalists used the translation bot again to collect and examine more than 1,000 photos and videos from government sources, plus more from Times and wire journalists on the ground. The images often showed multiple weapons and journalists identified more than 200 cluster munitions, widely banned in many countries, and more than 2,000 unguided munitions, which can kill and maim indiscriminately. Hundreds were found in civilian areas — evidence of possible war crimes. These records were cross-referenced for verification. This sometimes involved locating landmarks, like statues or signs, and then matching them on Google Maps or in other public records, sometimes with help from Google Translate and a digital Russian keyboard.

Context about the project:

Times developers faced technological hurdles in assembling the archive of official Ukrainian posts that provided much of the data underlying these two stories. Given the urgency of the situation, three software developers worked to understand the need, write integrations to Slack and Google Translate, and get the app deployed in the span of a day. The work resulted in a database of nearly a year of official social media posts in both Ukrainian and English, in a searchable format, unavailable elsewhere, which has helped both The Times’s live and investigative coverage of the war.

At the beginning of the invasion, a small team of journalists who had specialized in collecting, verifying and analyzing Covid and vaccine data quickly turned their attention to the war. This was an around-the-clock operation — similar to the early days of the New York Times Covid tracker — with team members working all day, including on nights and weekends. The translation bot brought in multiple posts per minute during busy periods in Ukraine, so it was important to figure out quickly which posts could be triaged and which were important to code into the database. The work needed to be done efficiently but meticulously, so a rigorous sourcing and fact-checking system was implemented immediately, requiring multiple sets of eyes on each image added to the database that eventually turned into the underlying system that powered the civilian targets story.

To identify the make and model of each munition in the weapons story, a journalist who formerly worked as a Navy explosive ordnance disposal officer painstakingly used several public and private weapons databases to systematically categorize each munition. This included systematically checking each item for unique markings, identification numbers and relative size and then comparing it with photos and schematics of weapons listed in the databases. In collecting visual evidence, reporters tried to catalog multiple photos of the same weapon from different angles, if possible, in order to help identify them more clearly (and to prevent counting duplicates). In the end, the number of cluster weapons and unguided munitions the team identified are undercounts because some weapons did not have clear enough markings in order for the journalists to confidently identify them.

What can other journalists learn from this project?

There is power (and speed) in having multiple journalists curate a dataset, but it is essential to establish a process to constantly check intercoder reliability (the extent to which multiple researchers agree on how to code the same content). The team members working on collecting this data worked remotely and it was crucial that communication was frequent and didn’t waste time. The team developed a system for quickly alerting each other and talking through sourcing and problems with a set of emojis. If a reporter saw a traffic cone on a post, that meant a teammate was working through the sourcing and others could avoid it. A big red “X” on a post meant it had been triaged. A checkmark on a post indicated that it was a duplicate. Emojis are often used for lighter purposes, but they were useful on this project as a way of visually encoding cues that everyone immediately understood.

That said, there is something to be gained by not being super efficient all the time. It is important to understand that even data journalism can cause trauma. Team members working on these stories spent weeks upon weeks examining photos and videos of the aftermath of often horrific attacks and then logging them into databases. Journalists found photos of families murdered in their cars, rockets crumpled in playgrounds and many other awful images. It was critical to continually remind the people working on these stories to take breaks or time off, if they needed to. The team also got into the habit of sharing more “hopeful” images they found with the whole group — this included a lot of photos of animals who had been rescued in Ukraine.

These stories exist because of a unique mix of automatically scraped data and hand-curated data. Both were crucial.

Project links:

https://www.nytimes.com/interactive/2022/03/23/world/europe/ukraine-civilian-attacks.html

https://www.nytimes.com/interactive/2022/06/19/world/europe/ukraine-munitions-war-crimes.html