The Fault in our Stats

Sibin Sabu

Denmark, the happiest country in the world also happens to be the second largest consumer of anti-depressants. Statistics can be extremely deceptive to the unaware. Depending on the statistic one chooses to use or ignore, contrasting images can be drawn before the public.

What can you infer from Crime Rate Statistics?

Kerala offers several interesting insights. Although known for its excellent social indicators and governance, it has the highest crime rate in the country. In 2012, National Crime Records Bureau pointed out that the crime rate in Kerala (455 per lakh of population) was more than double the all-India average (196). What is more interesting is that Uttar Pradesh (96) and Bihar (127) reported much lower crime rates.

Given by data alone, it appears reasonable to conclude that God’s own country, Kerala, is the most dangerous place to live in, while UP and Bihar are among the safest. The missing element here is that these conclusions are drawn from reported crimes and not the incidence of crimes. This breathes sense into an otherwise befuddling data.

Being more educated and empowered, Keralites are more likely to report crimes than those in other states.

The approachability and effectiveness of police force could be another reason. Similarly, the crime rate in Somalia (1.5), Iraq (2) and Libya (2.9) is minuscule compared to crime rates in countries such as Sweden (6456) and U.K (4447).

High crime rate may, therefore, be a reason to celebrate instead of being a reason for concern. Ironically, what we cannot infer from crime rate statistics is precisely what it’s used to define – the actual crime rate itself.

It appears paradoxical that the areas with high crime report low crime rate as per statistics.

Evidence-based policy making – A myth?

[su_pullquote]The ideological inclinations of policy makers may prompt them to be choosy in the use of statistics.[/su_pullquote]

The motivation behind collecting crime rate statistics might be to identify areas where the rule of law is poor so that corrective measures need to be taken. The government also collects a plethora of other data to evaluate the effectiveness of current policies and to frame the right policies going forward. The idea is to undertake evidence-based policies instead of ideology-based ones. The challenge, however, is in discerning the ‘evidence’ used in policy making. The ideological inclinations of policy makers may prompt them to be choosy in the use of statistics.

A couple of years ago, Arvind Panagariya who presently heads the NITI Aayog (which replaced the Planning Commission), caught attention for sharp criticism for being selective in his use of statistics when he presented the Kerala model as the Gujarat model in disguise.

Given the magnitude of studies available – often of contradictory nature, it has become easy to find or even generate evidence to force-fit one’s ideas and ideologies.

A 2013 study by CRISIL found Kerala, the southern state of India, to be the most equitable state in the country. Around the same time, National Sample Survey Organization (NSSO) came up with its report which found Kerala to have the highest economic inequality in India. The only significant difference between the two studies was in their methodology and it is astonishing to observe the impact it had on the results.

A Question of Motivation

[su_pullquote align=”right”]Perhaps the fallacy is in believing that statistics is objective and impersonal. Its subjective nature is reflected in how data is derived and interpreted. [/su_pullquote]

Perhaps the fallacy is in believing that statistics is objective and impersonal. Its subjective nature is reflected in how data is derived and interpreted. The appropriateness of methodology of the study or survey, quality of data cleaning, and the motivation of the individual(s) conducting the study are few factors that result in its subjective nature.

Think-tanks, which complement the government in policy making, are good examples. Government-run think-tanks cannot be expected to come up with studies which are critical of government policies. Reports will often side with the stance of the policy initiated by the ruling government. It is not surprising that Mr. Panagariya is an ardent supporter of the Gujarat Model.

Likewise, it is difficult to establish the credibility of reports published by think-tanks funded by corporates and private agencies. In recent years, several reports have surfaced suggesting that oil companies fund think-tanks to generate reports of climate change denial. Ideological leanings of the organization or individual conducting the study can also affect the nature and purpose of study undertaken, and the interpretation of data.

Monopoly and the concern over accuracy

Perhaps, the biggest challenge is not in the interpretation of data, but in guaranteeing the accuracy of data used for policy-making.

In January last year, the Indian government had introduced a new methodology to compute GDP. Several economists including RBI Governor Raghuram Rajan have expressed their skepticism regarding the subsequent growth figures. GDP growth has become a matter of national pride and is considered to be an indicator of good governance. Therefore, one cannot completely rule out the possibility that the dubious GDP data is a deliberate attempt by the government to create an ‘optical illusion’ of high economic growth.

The Grexit crisis exposed the fudging of statistical data by the government. Photo Courtesy: The Week

Governments have a monopoly over a lot of data, often for good reason. There have been instances when this monopoly power was exploited for its own vested interests – interests which were perhaps, not shared by the public it represented. If a public body feels that it can get away with misrepresentation of facts for its own survival, it might do so.

The Greece debt crisis should be still fresh in memory. The country admitted to fudging its budget accounts in 2001 to secure entry into Euro. What ensued later needs no reminder.

The message is clear. One needs to realize that statistics is a subjective tool. It is easy to fall prey to its deceptive power. Therefore, it helps to be skeptical and cautious when presented with statistics.

Ignorance of these nitty-gritties can have disastrous consequences and may lead to erroneous policy making. As someone said, acknowledging the existence of a problem is the first step towards finding a solution to fix it. Generating awareness and public debate would be the second.


Sibin Sabu is an academic associate at Indian School of Business, Hyderabad and specialises in Public Policy.