Are Black Swans BS?

Had a great discussion at the Breakfast Roundtable (CFO Core Concerns Conference in Boston) the other week. Mostly a collection of CFO's and CRO's (Chief Risk Officer) from a variety of companies. I especially enjoyed the question posed by Brenda Boultwood, CRO of Constellation Energy: "Was 2008 a failure of risk management or a failure of management to heed the warnings of risk management?"

While the media slammed risk management in general -- and VAR in particular -- I am more inclined to think that the risk managers have a good process that can spot systematic risks and cascading failures. In all the work I've done with Crystal Ball, it seems that the relatively easy part is for the analyst to figure out the issues and the hard part is driving organizational change. For some, seeing the range of possible outcomes given the uncertainty of the inputs, is one way to drive that organizational change.

Next up, Nicholas Nassim Taleb will be at the next CFO conference in Las Vegas, Sept 20-22. I'll be looking forward to his take on the role of risk managers in our now very uncertain world.

Teaching 5th Graders

Yes, I actually used this graphic to explain supply and demand to 5th graders. It has been a privilege to teach them the 'Stock Market Game' these past few Wednesdays. We decided to broaden the scope to include money and banking, entrepreneurship, accounting and happiness.

Happiness? Yes, happiness. When putting together the teaching materials, I had this feeling that we were feeding the assumption that more money means more happiness. So when we tested the kids today (with a really cool live clicker system) we asked this for the final question, "Can money buy happiness?" Yes or No.

No beat Yes 4:1. After some discussion, we concluded with "There is more happiness in the getting, than in the having."

Create Internally Consistent Goals and Plans: 'The Rope'

A challenge that all organizations face is setting goals and creating plans that are internally consistent. For example, if you were to estimate next quarter's revenues by product would it match you estimate for revenues by geography?

What I often find is that there are 6-8 relevant dimensions to estimate a number. When you 'think in ranges', each dimension gives a different range for that particular forecast. By overlaying these forecasts on top of each other, you get a visual depiction of your internal consistency.

If you were perfectly consistent, the forecast would all exactly overlay one another. On the chart, I've overlayed cumulative forecasts. A cumulative forecast shows the likelihood that the true outcome is at or below a given value. I've also added the Median estimate for each forecast.

What we can do is analyze why one method give a different range than another. For example, if our estimate by product is much higher than our estimate by geography, we can drill in and reconcile the two. Through this iterative process, you can harmonize your estimates. As the forecasts get closer and closer, this is what I call the rope. The goal is to have a tight, strong rope.

Finally, I like to look at the dispersion of the means. This give me a 'center of gravity' feeling for where the true number will likely end up. For the true value to be outside of this range, you will have to have been significantly wrong on several dimensions at the same time. If you have to give someone 'the number', the mean of the means is one way to resolve these ranges back to a single number.

What is the Probability of acheiving my ROI target?

ROI: Will this investment meet my ROI requirements? This question comes up every day in business. Every vendor has their version of the compelling ROI. The question the buyer should ask is the probability of a given ROI scenario.

Many corporate purchasers think of ROI in terms of hard dollar v. soft dollar savings. Hard dollar savings are those things that reduce my weekly, monthly or quarterly cash flow. Soft dollar savings do not. A classic example is 'you will save 5 man days per month of labor on this process'.

Many buyers use only hard dollar savings in their ROI calculations. The soft dollar benefits are just the icing on the cake. A typical target is to have a 12 month payback only counting the hard dollar savings.

With uncertainty management, it is straightforward to learn the probability of each scenario. In the chart above, we can see there is an 83% chance that the hard dollar ROI will be under our acceptable limit of 12 months. It is nice to also see that the worst case scenarios go out to about 18 months. Also we can see that if everything goes our way, then the payback could be as short as 4-6 months.

As a seller, you can be very persuasive to a buyer with this kind of analysis. As a buyer, it is essential to ask for this risk assessment to understand the probability of a vendor-generated ROI.

Fuzzy Dots are ... well ... Fuzzy. Let's keep them that way.

In College I studied Philosophy and Accounting. An odd combination, but there is an intersection with regard to Fuzzy Dots.

In Philosophy class, we learned that if a dot is truly fuzzy, not just blurry because of your eye piece, then no matter how fine a microscope you use, it will always be fuzzy.

Over in the business school -- and now in the business community -- I see a lot of business analysts trying to get better microscopes to increase their resolution and decrease the fuzziness around their dots. Now, if you are doing accounting of historical numbers, this may be a good idea. There is one true number that you can solve for. However, in many cases, the number the analyst is pondering is itself a fuzzy dot. Estimating next quarter's earnings, the NPV of a capital project, the ROI on an investment, the cost of a major project or the completion date a of major project are examples of numbers that are fuzzy dots.

To resolve them to a clear dot is to ignore their underlying reality which only creates an illusion of certainty. Unfortunately, many enterprise planning applications require the analyst to put in a 'naked number' for a fuzzy dot rather than a more realistic probability range. Worse, many managers require their analysts to just give them a number. See "Why Can't You Just Give Me the Number?" (Patrick Leach, 2006) for an excellent summary of this issue.

Many would prefer to have the illusion of accuracy (the NPV will be $20.687M) fostered by false precision. The only thing we know about such numbers is that they will be wrong: we just don't know by how much and in which direction. For further reading, see Phil Rosenzweig "The Halo Effect" (New York: Free Press, 2007).

While we are at it: What's Wrong with Benchmarking Competitors?

In recent conversations with CFO's the topic of benchmarking comes up. Of course the conventional wisdom is that this is a good idea. I've never (almost) done it and here is why.

First, who are your going to benchmark? If you are an airline, do you benchmark other airlines? That would be the straightforward answer. But what does this tell you? That your labor cost per flight mile is x v. y? What is actionable about that report? 9/10 it is not telling you something you didn't already know. Does the competing airline have the same strategy? If you are the premium service, global airline, would you compare yourself to the low cost, new entrant carrier?

All of this is generally beside the point, because your competitors are not other airlines. Perhaps it is other means of transportation (driving, trains, bus, etc). More interestingly, it is either someone completely not on your radar (Skype - free web-based video conferencing) or 'good enough'. "Good Enough" always seems to be competitor No. 1, but I've never seen that entity on a benchmarking report.

For an excellent analysis of your competitive dimensions, read and digest Clayton Christensen's "Innovator's Solution". Thinking through the Functionality, Reliability, Convenience and Price dimensions will change how you view your business. Understanding who may be a disruptor to you -- and to whom you may be a disruptor -- is an invaluable perspective. And makes benchmarking conventional competitors seem a little quaint.

So What's Wrong with Scenario Analysis Anyway?

When I meet with CFO's, I like to ask if they are doing scenario analysis. Because they all went to good MBA programs, they universally say "yes". I don't think that is a good idea. Let's explore what happens.

First, your team meets for a planning offsite at a nice location. So far, so good. Then among the SWOTs and whiteboarding session you come up with a list of key drivers for your business. There are always at least 4 or 5. Each driver has two or more possible values (either we are in a high regulatory environmnet or lax; competitive issues are increasing or decreasing; pricing power is high or low; channel effectiveness etc). With that many variables in play there are 20+ possible scenarios to consider.

Next, the group narrows the list of scenarios to a 'plausible' 3-4. (uh oh). The plausible few get the full integrated operational and financial planning process applied to them.

Then, reality hits. One of the implausible scenarios plays out. At a recent meeting with an airline CFO, he said when we were doing our planning the main concern was fuel cost -- we never considered a scenario where the price of oil is under $40.

Another way to approach this situation is to not try to guess as to which scenarios are plausible. In eliminating all but a few scenarios, you have just thrown out the most valuable part of the analysis: the rare, but highly-impactful events in the scenarios in the 'tails' of your expectations.

By keeping all scenarios, and using simulation to model the uncertainty in your key drivers, you can get a more realistic view of your entire assessment of the future. While no one can predict the future, with simulation you can better anticipate it so that you can be prepared for when the scenarios in the 'tails' play out.

Scenarios v. Strategies v. Simulation v. Sensitivity

In speaking with a variety of CFOs and risk professionals recently, it seems that there is come confusion over these terms. So before we go any further, let me clarify how I think about these terms. In any business decision, there are things you control (decision variables) and things you cannot control (assumptions).

Scenarios: A comprehensive set of assumptions that tell a consistent story. For example, Obama gets elected, price of oil is under $50, dollar is weak, demand is soft, labor is cheap, etc. would be one scenario. McCain is elected, oil is at $150+, strong dollar, etc. would be another scenario and so on. Scenarios show one possible outcome at a time.

Strategies: A comprehensive set of decisions that tell a consistent story. For example, we will be the low-cost, hi-volume provider in our market using a fast-follower product strategy. Another strategy may be to conserve cash, reduce capicity, protect margine etc. A third may be to spend agressively to gain share.

Sensitivity: This is the worst-best-most-likely scenarios.

Simulations: Simulations simultaneously test all assumptions across a broad range of possibilities. A simulation yields the probability of any particular outcome. This is what Crystal Ball does.

So once you start 'Thinking in Ranges'; the next key step is to understand what is driving that variability. If you are an executive making decisions based on numbers, then you should always:
1. Know the range around the number, and
2. Know the key drivers of that range.

A sensitivity chart shows you those drivers. It much more realistic than a tornado chart because it is varying all of the assumptions simultaneously and each assumptions varies according to its own distribution.

Sensitivity charts are also handy for negotiation and expense cutting. When negotiating, you know which dimensions you can compromise on and how much while still preserving your overall objective. With expense cutting, you can trim expense in low impact areas to preserve your upside.

I like to think of these as "Know What Matters" or "Where to Focus" charts. And every organization needs help in understanding where to focus its energies.

So this is where it all began. I first saw a Crystal Ball forecast chart in the summer of '94 and it changed how I thought about the world.

Thinking in Ranges. No longer could I just see a number, but rather I could see the range around the number. How different would it have to be for me to make a different decision?

When I received an MBA, I especially liked spreadsheets and NPV calcs. Of course, the whole point of NPV is what is the probability that NPV>0? With just Excel, you can only see Possibilities, not Probabilities. With Crystal Ball you can see that left, red tail. That is the key to understanding a project's risk and uncertainty.

Uncertainty Management

Having been involved with UM since 1994, I thought I'd share some thoughts on the recent focus on these 'uncertain' times.