Media Briefings

The Questionable Reliability Of International Statistics: Evidence From The Human Development Index

  • Published Date: June 2011

Small revisions to the data that go into the widely cited Human Development Index (HDI)
can substantially alter a country’s position in the global rankings produced by United
Nations Development Programme (UNDP). That is one of the findings of research by
Professor Hendrik Wolff and colleagues, published in the June 2011 issue of the
Economic Journal.
Their study, the first comprehensively to measure data error in international statistics,
reveals how such errors can lead to inappropriate policy conclusions, particularly as a result
of the UNDP’s classification of each country into one of three ‘development bins’.
The researchers urge the UNDP to discontinue this practice because the cut-off values
seem arbitrary, the classifications can provide incentives for strategic behaviour and they
have potential to misguide politicians, investors, charity donators and the public.
The HDI, published annually by the UNDP, has become the most widely used measure for
communicating a country’s development status. Compared with the traditional measure of
GDP, the HDI is a broader measure of development, since it captures not only the level of
income but also incorporates measures of health (life expectancy) and education (school
enrolment and literacy rate).
Importantly, the UNDP ranks all countries into a global league table and then classifies
each country into one of three development bins:
 Low human development for HDI scores between 0.0 and 0.5,
 Medium human development for HDI scores between 0.5 and 0.8
 High human development for HDI scores between 0.8 and 1.0.
Each year when the new HDI statistics are published, much public attention focuses on the
ranking of a country relative to its ranking in previous years and to those of competing
For example, in 2001, when Canada lost the top position, The National Post wrote: ‘We’re
not No. 1! Canada drops in UN rankings… Prime Minister Jean Chretien often refers to the
report in public statements and speeches’
. Or in 1998, when Pakistan (ranked 138 and 119
the previous year) bypassed India (ranked 139 and 118 the previous year), The Tribune
noted: ‘Pak beat India, both lose!
Hendrik Wolff first gathered data for the HDI when he was working in Ghana in 1997. In
later years when he wanted to compare the rankings, he noticed that the original ranks
jumped around substantially, which led to him and his colleagues to scrutinise the statistics
used in the HDI.
Their study addresses these questions by exploiting (1) the originally published HDI time
series; (2) the variables used to construct the HDI; (3) changes to the HDI formula; and (4)
documented data revisions. It identifies three sources of data error: measurement error due
to data revisions; data error due to formula updating; and misclassification due to
inconsistent cut-off values.
The research finds that:
 The HDI contains data error standard deviations ranging from 0.03 (the United
States) to 0.11 (Niger), which is significant given the 0 to 1 scale of the HDI.
 The magnitude of the error variances is greater the lower the HDI rank. This is
consistent with the quality of the statistical agencies improving with higher
 11%, 21% and 34% of all countries can be interpreted as misclassified in the three
development bins due to the three sources of data error, respectively.
 The expected deviation in rank in the world HDI league table is nine rank positions.
These calculations show that statements such as ‘Pak beat India’ are to be
interpreted with great care.
 By replicating prior academic studies, the research shows that key parameters vary
by up to 100% due to data error.
The authors suggest that the United Nations should discontinue the practice of classifying
countries into the three development bins. Politically sensitive uses of the HDI might
potentially provide perverse incentives for a country to manipulate the sub-indicator
variables, if it has realised the comparative advantage of a 0.49 HDI score versus a 0.51
Some of the criticism raised by this research were noted in The Economist in January 2011: The UNDP
responded that it undertook a systematic revision of the methods used for the calculation of
the HDI and that the new methodology and classifications of countries are now robust:
Notes for editors: ‘Classification, Detection and Consequences of Data Error: Evidence
from the Human Development Index’ by Hendrik Wolff, Howard Chong and Maximilian
Auffhammer is published in the June 2011 issue of the Economic Journal.
Hendrik Wolff is at the University of Washington. Howard Chong is at Cornell University.
Maximilian Auffhammer is at the University of California at Berkeley.
For further information: contact Hendrik Wolff on +1-510-220-7961 (email:; Howard Chong on +1-510-333-539 (email:; Maximilian Auffhammer on +1-510-463-1546 (email:; or Romesh Vaitilingam on +44-7768-661095 (email: