ON March 23, 2015, the world bid farewell to a true statesman in Lee Kuan Yew, an iconic human being and one of the greatest politicians of the 20th century. All around the world, people paid tributes and crafted obituaries to Lee, focusing mostly on his achievements in the development of Singapore as a first-world nation.

This article does not seek to recap those tributes and obituaries; rather, it raises an important question in discussing Lee’s legacy — what is the counterfactual?

Briefly, in social science speak, the counterfactual scenario is the scenario that would have happened had the “treatment”, “programme” or “policy” not taken place. Dan Levy, a senior lecturer at the Harvard Kennedy School, gives a personal angle on the counterfactual. He asks his students, “What would you be doing if you were not at the Kennedy School? That is your counterfactual.” Similarly, each reader may ask himself or herself, “If I were not at my present job right now, what would I be doing?”

The reason the counterfactual is important to know in any empirical research project, especially in the social sciences, is that it gives us a basis to compare outcomes of policy. For example, take Teach for Malaysia’s fellowship programme, which places highly-qualified young “fellows” as teachers in schools in underserved communities for a period of two years.

Consider, for instance, the outcome of average examination scores of students in classes in which a Teach for Malaysia fellow teaches. For us to conclude that particular fellow was effective in his or her work, we should not merely look at whether the examination scores for the students increased, but by how much more they increased relative to the examination results of those students had they not been taught by that Teach for Malaysia fellow. The latter is the counterfactual.

An astute reader would notice something very peculiar, but very true. In real life, we can never observe the counterfactual. We can only observe what happened, and make conjectures on what would have happened.

This is the primary problem in empirical social science research on programme evaluation — known as the identification problem — and there is a wide variety of ways that economics methodology (econometrics) has introduced that try to replicate the ideal counterfactual. For the purposes of this article, those methods are relatively less important than the general idea of evaluating some outcome against its counterfactual.

What does this have to do with Lee? Well, the true measure of how great Lee’s achievements are is the difference between the Singapore as we know it today and the Singapore in which some other prime ministers helmed the development of the island nation. In other words, we need to compare how Singapore is today with how Singapore would have been under the stewardship of an average leader.

In basketball, there is a term for this, called “Wins Above Replacement Player” or WARP. WARP tells us how valuable player X is by measuring the number of wins achieved by a team in which X plays (alongside four other average teammates) versus the number of wins that would have been achieved by a team in which a replacement-level player (average player) replaced X.

If some other average player would have generated the number of wins that you generated for your team, how valuable can you really be? On the other end of the spectrum, replace Lionel Messi in Barcelona with some average attacking player like Adam Lallana and watch Barcelona battle it out for a Champions League spot in Spain’s La Liga.

Turning to the case of Malaysia, when we evaluate government policy, we should not evaluate the efficacy of policy simply by what happened, but rather by the difference between what happened and what would have happened in the absence of that policy.

Consider, for example, the Economic Transformation Programme (ETP). On Pemandu’s ETP website, it states that Malaysia’s Gross National Income (GNI) per capita has increased from US$6,700 in 2009 to US$10,600 in 2013, on track to achieving the US$15,000 required to become a high-income nation by 2020. That Malaysia’s GNI per capita has increased is very clear, assuming the statistics are right. What is less clear is what the ETP had to do with it.

If Malaysia’s GNI per capita was going to increase from US$6,700 to US$10,600 without the ETP anyway, then the ETP has been, overall, underwhelming. The government cannot prove the ETP’s effectiveness by simply quoting how much GNI per capita increased. The proof of the ETP’s efficacy is how much higher GNI per capita is today versus a scenario in which there was no ETP.

If the economy was going to grow at 5% anyway with or without the ETP, why waste time on the ETP? A rising tide lifts all boats. This is also true of Government Transformation Programme indicators. What we need to know before continuing to commit to the ETP and GTP is whether they have added value above and beyond what would have happened in their absence.

It may seem tautological to say that the common is more likely to occur than the uncommon. Yet, it is a concept that is not well-grasped by many, especially those with an agenda to push. Of course the CEO who saw a 30% increase in firm revenues is going to say that it was because of him. But we should think more deeply. If another average CEO took his place, would revenues also have increased by 30%? If the answer is yes, then that CEO deserves no additional plaudits than the average. If the answer is no, then he deserves his bonus.

However, given that the common is more likely to occur than the uncommon, we should expect, on average, everything we observe in terms of outcomes to norms rather than exceptions, just by cold, hard probability. In statistics, occurrences that are significantly different from the status quo (either positive or negative) occur only 10% of the time.

This means that, 90% of the time, we should expect to see outcomes that are not really different from the average outcome. This is true of examination scores, of goals scored, of firm performance, of policy outcomes and of life. Going back to where we started, we can ask if Lee was an outlier as a politician or was he not significantly different from the average politician.

I would like to suggest that in the case of Lee, he was statistically significant. He was the exception rather than the norm, and the world is poorer without him in it. His choices and his actions worked because he was who he was; those same choices and actions, cited as models of governance by many, may not work out as well in the hands of another politician, who — as we know — is likely to be a merely average politician with 90% probability.

Nicholas Khaw is an economist-in-training at Harvard Kennedy School. Prior to this, he was an assistant vice-president of the research division of Khazanah Nasional Bhd.

This article first appeared in Forum, The Edge Malaysia Weekly, on April 6 - 12, 2015.

Save by subscribing to us for your print and/or digital copy.

P/S: The Edge is also available on Apple's AppStore and Androids' Google Play.