What If?

The point of simulations, and models in general, is to get some understanding about what will happen if we try something. By understanding the consequences of our actions, we can optimize them to have the best chance of getting the results we want. The process of doing this is called scenario analysis. Here’s a short tutorial and an example “ripped from today’s headlines”.

Scenario Analysis
Scenarios tell us what will happen if factors outside our control act in our favor, or against it. These factors might be specific competitors or adversaries, the inanimate world (e.g. natural disasters), or the emergent behavior of the collection of others just like us (e.g. the economy). Whatever the nature of these external influences, we want to know how these will affect our plan and the attainment of its goals.

Our ability to forecast the future using models and scenarios depends on three things:

1. The accuracy of the model
2. The amount of uncertainty in how we expect influencing factors to behave
3. The number of influencing factors we are including

So, even if we have a perfectly accurate model for history, we can't have a perfect forecast because of #2. Furthermore, any model we create is an simplified approximation of the real world, which guarantees that (#1) our model is not perfect and (#3) we will omit some influencing factors.

Though not perfect, scenario analysis allows us to develop

• a sufficient understanding of the risks and rewards of a several alternative courses of action,
• an understanding of what we need to watch out for, and
• contingency plans in case a problem arises

In his now classic book The Art of the Long View, Peter Schwartz describes the purpose of scenario analysis as the systematic exploration of several possible futures. Schwartz notes that scenarios are tools "for ordering one's perceptions about alternative future environments. The end result... is not an accurate picture of tomorrow, but better decisions about the future." No matter how things might actually turn out, both the analyst and the policy maker will have "on the shelf" a scenario (or story) that resembles a given future and that will have helped them think through both the opportunities and the consequences of that future. Such a story "resonates with what [people] already know, and leads them from that resonance to re-perceive the world... Scenarios open up your mind to policies [and choices] that you might not otherwise consider."

There are a number of techniques for scenario analysis. A few of the more common ones are described below.

Worst-Case Analysis
You describe what you think is most likely to happen (known as the “baseline”), and what will happen if things go really bad (the “worst case”). Of course, things can go really well too, but that’s a situation we don't usually spend a lot of time analyzing.

Monte Carlo Analysis
You identify all the variables that influence your results, and the likely range (including good and bad) that they will cover. You then generate a large number of scenarios (hundreds to millions, depending on the number of variables) by picking values at random for each of the variables. You then evaluate your plan for each of the scenarios, average all the results and call that the “expected value.” If you compare the expected value (the return) and standard deviation (the risk) among multiple plans, you can choose the point you want to be on in the risk/reward spectrum. Typically a higher expected value has more risk associated with it.

War Gaming
This is typically a multi-player and turn-based situation in which real people take on the roles of competitors or adversaries (hence the “war” part of the name) and compete with each other in a virtual environment. This is an excellent approach for strategic decisions, because real people who are experienced in a domain (a market or a theatre of war) will pose much more realistic challenges to your plan. Furthermore, the turn-based aspect of war gaming means that the players can adapt to changes.

Vignettes
In this approach, people work together to identify the factors that can affect a system. These drivers are put into a common framework, which might be conceptual or a concrete software model. There are usually too many variables (influencing factors) initially, and the team works to pare this list down in two ways

1. Developing a consensus as to which variables are independent (as opposed to variables that directly influence each other)
2. Determining which of the variables from #1 are most influential in the behavior of the system.

Once there is a reasonable number of variables, the scenarios are fleshed out. Usually the variables are boiled down to two, which form the x and y axis of a plane. Each quadrant of the plane is given a name. For example, the x axis might range from dictatorship to democracy, and the y axis from socialism to capitalism, giving you a socialist dictatorship, a capitalistic democracy, and so on.

With these scenarios defined, the team discusses what the implications are in these situations. What opportunities and risks arise? Which do we want? How do we get there? Etc.

A Relevant Example

The current economic crisis provides a great real-time case study of scenario analysis. This is because the US government will, over the next several weeks, be subjecting major banks to a “capital assessment”, more commonly called the “Stress Test”. This is a “What If?” analysis in which:

"Each participating financial institution has been instructed t o analyze potential firm‐wide losses, including in its loan and securities portfolios, as well as from any off‐balance sheet commitments and contingent liabilities/exposures… The capital assessment will cover two economic scenarios: a baseline scenario and a more adverse scenario."

The two scenarios are known as “baseline” and “more adverse”. The baseline scenario “is intended to represent a consensus view about the depth and duration of the recession.” The variables in the scenarios are GDP (Gross Domestic Product, a measure of the total economic output of the US), unemployment, and housing prices. The more adverse scenario looks at situations that are believed to be 10% likely for each individual variable, and assumes that all three variables fall reach this pessimistic value. The charts below show the three variables for each metric.

The government chose worst-case analysis for a few reasons.

• First of all, evaluating a $100B bank’s balance sheet against a scenario is a very involved task, and time and manpower is short. So, the number of scenarios needs to be limited and Monte Carlo is out.

• Since the government is the only “agent” that can take any action to stabilize the economy, there isn’t really any competitor or adversary (other than the “invisible hand”). So war-gaming is out.

• Finally, there’s no need to explore where the system (the economy) might go—that’s pretty well known. The influencing factors are well known too. So no need for Vignettes.

What will the government do when the stress test is complete? Stay tuned as we all find out…