Modeling – A Leap of Faith Amidst Uncertainty?

In this pandemic, a lot has been said and written about models – their usefulness, their limitations, their inaccuracy and more. If nothing else, we have become familiar with epidemiological models – at least, the ‘curve’!!

The most famous model being used today is the COVID-19 Projections developed at the Institute for Health Metrics and Modeling at the University of Washington. Public health experts have disputed its forecasts, even claiming it is purely statistical and has not epidemiological basis. Other models have been developed at the Northeastern University, Los Alamos National Lab, MIT, Imperial College London, Columbia University. These models use different techniques and assumptions, leading to differing projections about the trajectory of the pandemic. Some of them have been consolidated at the University of Massachusetts Amherst – one can clearly see the varied projections.

Source: The New York Times

At the same time, these are being used by policymakers in making decisions and by others to criticize those very decisions because of the disparity in these models. For example, the New York State, these models predicted, would need much more ICU beds and ventilators than it has actually turned out to need. Does it mean that the models were wrong? Or has the social distancing policies guided by those models worked such that less of these medical equipment are needed?

We don’t know yet.

So, what is a model?

A model is, by design, a fragment of reality – developed to study specific features of it. It explains some specific aspects of the world, not the whole. It has assumptions built into it, that are reasonable in some contexts, not so in others.That is why we have many different models, even for the same narrow issue we try to understand. Features of the system that are important today maybe redundant tomorrow. What is crucial for some people may not be so for others.

Another reason we have so many models is there is a healthy disagreement among those who develop these models. So each model has its strengths and weaknesses.

The question then is, how should judge at a model?

An initial question to ask is: What is the model trying to explain? Looking for an answer to a question that is not the focus of the model defeats the purpose, doesn’t it?

A second step might be to see if the assumptions built into the model are reasonable for the question it is trying to answer. Does it explain the reasoning behind those assumptions? Are those assumptions intuitive? Do they align with what is already known about the world?

Next, we should look at the specification of the model itself. Given the question it is trying to address and the assumption it makes, are the elements of the model logically plausible? Does it explain the intuition behind each new step? Does the explanations it provides make sense intuitively to someone with some idea of the subject at hand?

Finally, does the model fit the data reasonably? To what extent? Does it address reasonably well, the limited question it was trying to address? Does it leave open possibilities to be tweaked to a different context? Can it be adapted as we get new data, wouldn’t that be fantastic?

So, what are we left with?

Hopefully, more understanding, both about the pandemic and the practice of modeling itself. We would be better off realizing that different models explain different things and hence, we should look at a multitude of them before we make up our mind.

Is uncertainty the only certainty? At least for now. In time, we will have a clearer picture. About this pandemic. Then, we will have a different uncertainty. Which is uncertain.

Policy studies in India: Finding direction and purpose

In recent years, there has been a steady increase in the number of think tanks and academic institutions studying public policy in India. These range from government funded bodies (like NITI Aayog), private sector organizations (like Observer Research Foundation or Centre for Policy Research) to international forums (like World Economic Forum, World Bank) and policy schools in premier higher education institutions (Centre for Policy Studies at IIT Bombay, School of Public Policy at IIT Delhi).

These indicate a divergence from the previous dominance of the Civil Service, mainly through the Planning Commission, in policy recommendation and analysis. While implementation was seen as the key determinant of policy success or failure till 1990s, this has given way to analysis of policy design, assumptions, forecasts and the very objective of the policy. This has also resulted in both a politicization of policy and the policy-fication of politics. In spite of the inchoate nature of the process, policy debates still provide the core of ideas, justifications and concepts – although more contentiously in a political environment where achievement of power through ideological hegemony seems to be the predominant goal. This articles discusses the direction of policy studies and its changing relevance in current political context.

While Planning Commission in India signified policy as a technocratic tool of an enlightened administration, recent developments have taken an argumentative turn locating policy both as a strategy servicing politicians and interest groups (politicization of policy) and as a tool to structure and systematize the public debate between experts, citizens and states (policy-fication of politics). The Commission in its latter decades was criticized for its lack of political realism and introducing political preferences under the guise of neutral procedures and technicalities in pursuit of political objectives.

Although similar critiques can be made of Niti Aayog, today’s broader policy debate recognizes human biases, political motivations and power dynamics much more than it did in the past. Diverse and emerging institutions of policy studies also reflect reduced relevance of traditional political scientists in the power hierarchy of policy process. This is signified by shift to a more technocratic approach that studies impact, processes and content of public policy based on causation, falsification and evidence. This seems to be a more causal approach that diagnoses problems, conducts trials/experiments and predicts impacts of policy interventions. To what end, is an open question. They may be just as political in the guise of neutral.

Moreover, there is an added appreciation of action imperatives and political demands that policymakers face, although sometimes it may be used to justify very bad but enthusiastic policies. Policy studies today lie at the intersection of scientific rationality as a means of solving collective problems and the socio-cultural fragmentations that regard rationality as exclusionary, undemocratic and incompatible with diversity, and hence fallible. To ameliorate this, policy reports try to underline improvements and modification as important components of policy, thus locating it in an iterative social context of public understanding, dialogue and action.

At the same time, opening up of the policy process has also lead to multiple cosy relationships among politicians, administrators, analysts and commentators who have coherent views on an issue. This generates pockets of influence with divergent political framing systems, whose relevance changes with power dynamics. In fact, this is a clear example of politicization of policy where any evidence is no more than an argument to further an outcome.

In this context, policy studies may be seen just as a systematic means to provide clever strategic shortcuts and simplifications to decision-makers with only modest changes in their knowledge, i.e. policy analysts are seen as just providing ammunition in a rhetorical contest whose policy outcome has been decided by those in power. Optimistically, this can also be viewed as a way for to forge common ground between competing interests. However, this may also create a moral relativism where reprehensible policies suddenly emerge as solutions from the supposed consensus of participatory or electoral politics.  It disregards the conditions for such a political consensus, if it can be so called, resulting in political deception and manipulated legitimation of forced consent.

One of the issues with this argumentative turn in policy studies is the creation of counter-experts immune to learning or reflection – ‘tribes of experts’ – who create ‘contradictory certainties’ beyond comparison for politically persuasive audiences, which reinforces polarization and leads to policy paralysis. Policy studies today is caught between the practical demands of scientific analysis and the increasingly tenuous practice of politics. It is in a dilemma between serving either an active participatory, national citizenship, or a self-proclaimed, enlightened, policy-making political élite (includes opposition) which is global.

This predicament also signifies a gradual decoupling of policy studies from its previous role in supporting government-initiatives, towards shaping debate on issues that have either skipped decision makers or which require more global agreements to emerge before a policy problem is even defined. While policy studies is an emerging field in Indian academia, it would be wise for its promoters and practitioners to recognize the direction they are treading on and reflect on the way forward. The fractured nature of modern politics can easily seep into policy studies, undermining the expertise of policy analysts and degrading the quality of policy-making.

References:

Hoppe, R., “Policy analysis, science and politics: from ‘speaking truth to power’ to ‘making sense together’, Science and Public Policy, 26-3 (1999)