Precision v. Accuracy: How about using a range?


The concept of precision verses accuracy was first brought home to me back in the depths of time while I doing Physics "A" Level in high school and calculating measurement errors in experiments. From this I learned that no matter how precise one's measurements were, there an inherent error in the tool which was used to measure the results. Taking into account that lack of precision provided more accurate results, which is what we sought.

When I got involved in finance during my time in business school, the measurement tool was currency, and so precision was deemed to be the smallest currency unit, i.e. a cent. This precision was encouraged by the wonderful computing power of Lotus 1-2-3 and then Excel which allowed you to do forecasts that were precise to the cent going out years based on a few assumptions. While the precision was there, accuracy was foregone as we built huge models on a few simplistic assumptions.

However, upon leaving business school and entering the real world my illusions were quickly dashed. Having been at work less than a week, I presented my boss with a 10 year forecasting model which showed profit and loss statements, balance sheets, cash flows statements and of course a discounted cash flow valuation. He looked at it and asked how confident I was in my analysis. As I defended it, he looked at me and asked in a sort of Baynesian way, "Would you bet your bonus on company making the revenue numbers you have for this year?" (we were a quarter of the way into the year). 

"No!" was my quick reply. 

"In that case, how can you defend ten years of numbers which all build off that initial number?" he asked. It hit me and that has lived with me ever since that we quickly get sucked into the precision of our models, but really the numbers are so inaccurate.

During my career, I have see precise projections for sales, projects, costs being bandied around in companies by everyone, including the CFO. Yet few, if any, would bet a modest amount, let alone their bonuses, on those numbers being the final results. Thus we end up with the corporate situation where everyone knows the numbers are wrong but cannot admit it.

However, when working with these same companies and trying to get them to accept the concept of ranges in forecasts and probability distributions, I am regarded as though I had just stated that the earth was flat and supported by elephants. One CFO of a public company looked at me with total distance and asked "Why would anyone ever need something like that?".

When I responded that I was seeking more accuracy, albeit with less precision, in financial projections it was obvious that had I reverted to speaking Fanagalo. Another CFO used precise numbers (down to the dollar when their revenue was in excess of $20 million, and none of their forecasts were ever correct but he could not accept that forecasting to the million was acceptable. I heard that their last results were a huge surprise given the forecasts; however given the forecasting process I was not surprised at all.

This trade off of precision in return for greater accuracy continually is rejected by many of the financial professionals that I meet. I have wondered if the cause could be the focus on precision encouraged by the accounting profession that make it hard for these professionals to let it go.

So going back to my boss' logic - if you would not bet on a single number being accurate, would you be prepared to bet on a range? For example, the forecast may say that next years' revenue will be $10 million, but when pushed no one will bet their bonus on it. So would you be willing to be that it will be between $9 and $11 million or $8 and $12 million? When you reach a range that you would be willing to bet on, then surely that is the range to use in the forecasts.

Using ranges like this and applying techniques like Monte Carlo simulations results in far great accuracy in forecasts and an understanding of the risk profile of the results, i.e. there is a 50% probability that revenue will between $9 million and $10 million or that profits will between $0 and $1 million. Furthermore one can see what items are key to the final results what is not. Thus company's can focus on those key items rather than getting distracted by the noise.

It is best to remember Nils Bohr's quote "Prediction is very difficult, especially if it's about the future." Therefore, to make the predictions more useful, it is best to forego precision in return for greater accuracy.