Another Index. Armenia is #95. Meaningful?

The Legatum Institute, a London-based think tank, released its annual “Prosperity Index” and, as reported by PanArmenian, Armenia came it at #95.

After looking through the data, I can say that it is varied enough that it presents what I’ll call a picture of general understanding. That is, the use of reports like these is less what rank a country achieves and more about giving the reader an idea of where the country falls on a wide spectrum. In the case of Armenia, the prognosis looks to be that it has a lot of room for improvement.

Otherwise, the “index industry” (c) William Bairamian, is in full swing. Every think tank and their mother (do think tanks have mothers? the jury’s out but I’m going with yes) is publishing an index of something: happiness, corruption, poverty, water, sex, environment. Some are useful, some are an exercise in finding data no one else has yet manipulated and putting it in digestible form to keep up with the think tank Joneses. The Legatum report falls somewhere in the middle.

They aggregate dozens of pieces of data, including polling data from Gallup, so the ranking gives a more holistic picture than something like “X is the happiest country in the world and Y is the unhappiest based on our evaluation of data about the average distance a citizen must travel to the nearest lollipop stand.”

However, strongly lacking is any non-quantitative, country-specific analysis. The whole report is neatly based on an evaluation of numbers but an overreliance on numbers doesn’t tell the whole story. They do recognize this, though insufficiently, in my opinion. Their section on Prosperity Index Anomalies grants the inconsistencies that the data may belie; it deserves more attention.

Here are a few of the issues, in order of importance:

1) Lack of qualitative analysis

This is a major problem with these indices: everything is reduced to data. Culture, for example, has no bearing on the evaluation of the results, probably because it cannot be dependably quantified.

To make the point, here is a scenario:

You’re Armenian, you go to someone’s house and they ask you if you want something to eat. Almost invariably, you will refuse the first, maybe, thousand times they ask.

This cultural peculiarity in answering questions is not represented in polling data although it may well significantly skew responses in one direction. And when a significant part of your conclusion is based upon polling data, this can have a huge impact when that information is whittled down to a number. 95, perhaps?

2) Autocratic countries

As noted in their Anomalies section, it’s hard to conceive that people in an autocratic country might answer questions fully truthfully. A conversation might go like this:

Pollster: “Do you feel you’re able to freely express yourself?”

Citizen who believes he might be asked a similar question by undercover police to determine who isn’t saying the right things so they can get them to say the right things…in an empty room…with no cameras: “Obviously! We’re freer than hell! I mean, not just hell, everywhere! Long live our nation! Can I go home now?”

Pollster: “Of course you can. I’m just a pollster. I can’t prevent you from going home.”

Citizen: “Sure. Last time a guy told me that, I woke up without any clothes and pain in my, er, uh, my face! Because I was smiling so much! Have a nice day!”

Conversely, and curiously, when a country has greater freedom of expression, citizens may use that freedom to comment on the country’s lack of freedom more freely.

3) Unrecognized countries

This is an issue when, for example, the people of Artsakh aren’t polled. Comprising a sizable part of the population of Armenia, about 7-9%, it may have had an impact on the results, which Legatum seems to acknowledge.

Similarly, if they did the survey 25 years ago and asked Armenians living in Artsakh, then under Azerbaijan SSR, “How do you feel people of other ethnicities are treated in this country?”, I’m going to guess that would’ve affected their score a wee bit.

4) “Data Lag”

Not all the data is current. Some data points are from 2012 or, even 2011. Kind of a big deal for an annual prosperity index.

It’s not perfect. Nowhere close to it. But it has some uses. I think it’s better to explore the data that’s conveniently summed up in one place and beautifully presented rather than getting hung up on the ranking which, in a defiance of the laws of mathematics*, is less than the sum of its parts.