Tamil Nadu, the southern most State of India, is relatively one of the most liberal and progressive in the country. However, the recent anti-immigrant shift in the world politics percolated into the State’s policies and parties resulting in nationalist groups calling for “protection” of jobs and resources from “outsiders”. The effect such anti-migrant stance had in other States impressed upon the need to nip such sentiments in the bud through hard data and proving that the anti-migrant fear is a concoction of divisive politics. Thats what the story achieves. Using census data we proved that, not only are migrants in
Politically, none. However, the effect of such stories will only be known when the concerned issue gets big. As i had mentioned in the description, we chose to deal with the issue in the early stages. When we addressed the problem, only a handful of parties had issued such protectionist calls. However, the main opposition too had voiced such concerns, though in a mild way. If the issue gets big, like most of them do around elections, the hard data we published will come in the way and shed light on the ridiculousness of the claim: “migrants are taking away jobs in Tamil Nadu”. The data shows that migrants are not taking away but creating more jobs for the locals as most of them who come to the State are not seeking employment but have come for other needs. As the data is sourced from the Census, given the rationalist origins of the parties in the State, we believe that they will see sense and give up the call. However, the effect it would have had on readers is of far more value. If such parties continue with their anti-immigrant stance, we believe that readers who are also prospective voters, will neglect them in the elections. Which we feel is a prospective impact of the story.
Given the limitations of Census data dissemination in India, we were not in need of advanced tools or technologies. In India, most of the data is available in PDF files. Especially, some of the important documents may even be presented as image PDFs, thereby enhancing the difficulty of culling the data out. In this specific project, the Census data was available in PDFs and also they are arranged in a haphazard fashion. The challenge was to make our way through the maze of unorganised data and not so properly defined columns and make sense of the conclusions. Most of the data was culled out using Tabula and cleaned using microsoft excel. The definitions of columns were not available. Thus sources from Indian ministries and census departments were contacted and the definitions were confirmed. Then we used Tableau to visualise the scatter plots.
What was the hardest part of this project?
As i said before, in India, data is not collected or maintained in a uniform manner. The recent stories in Indian media about government hiding important public data (case in point the household expenditure data and the jobs data which were hidden from public and later leaked) proves the level of influence government has over its statistics department. This makes it a challenging task to fetch data in first place. The next part is about reliability of the data. Most of the column heads in the census data which we used, did not make sense. We had to call up the concerned departments, mail them to fetch those details. Most of them would not respond given the red-tapism in India. Thus, the project, which should have taken just over a week in the Western countries, took us a month to even understand what the data was really about. Then came the problem of ensuring whether the reality on the ground coincided with the data. Whoever we spoke to on the ground felt that the data was wrong. This speaks volums about the wrong perception people have about the number of migrants in the State. The non-belief in the data and the low levels of jobs-migration it was citing led to sources distancing away from the story, saying we are projecting something which was not a reality. This was in a way the success of the hard data which we pulled out, where perceptions were broken and reality was restored. No other Indian news organisation attempted this story despite common availability of census data.
Other than data collection and making sense of the data without proper column heads and ensuring its credibility given the very low levels of jobs-migration it was suggesting, the rest of the project was a breeze.
What can others learn from this project?
That, even in mundane data sets, lies the best of the stories. Also, public perception in a politically charged environment will often be skewed and only hard data can restore balance. That, when reporting hard data which goes against public perception, experts will scowl at the data and refuse to comment on the data as it is unpopular.