Big Finance’s hyper-focus on frequency creates a blindspot on magnitude; It doesn’t matter if your forecast is right 99 times if on the 100th time you are not only wrong, but catastrophically wrong

Big Finance’s hyper-focus on frequency creates a blindspot on magnitude

And what can be done to change that

By Shane Parrish | November 7, 2013

My main takeaway from Alan Greenspan’s latest book, The Map and the Territory: Risk, Human Nature, and the Future of Forecasting: Forecasters (and all those who rely on them) fail to realize that over the long run, even frequently accurate predictions mean little without taking into account the magnitude of relatively infrequent mistakes.“In practice,” Greenspan writes, “model builders (myself included) keep altering the set of chosen variables and equations specifications until we get a result that appears to replicate the historical record in an economically credible manner.”

Consider an example from the financial crisis. House prices did nothing but go up in value for a long time, so all of the models assumed that this would continue to be the case. This perfectly predicted the past, but not the future. As we now know, house prices can indeed go down.

Economic models are used to make policy decisions, set interest rates, and intervene in the so-called normal functioning of the market. When the models work, they work very well. When they don’t work, we’re left with a massive mess.

Greenspan, for one, places too much weight on the historical accuracy of models. And his plan for the future — essentially, more complicated and better models — is unlikely to avoid catastrophes like what we saw during the Great Recession.

“September 2008 was a watershed moment for forecasters, myself included,” he writes. “It has forced us to find ways to incorporate into our macro models those animal spirits that dominate finance.” But this still leaves us relying on these models, which will effectively forecast the future — until they don’t.

It doesn’t matter if your forecast is right 99 times if on the 100th time you are not only wrong, but catastrophically wrong.

Consider an investment. Let’s say a statistical model has helped your mutual fund generate 20 percent returns compounded annually for 20 years. Everything is going splendidly, and you have just one more year until retirement. In that last year, however, a historical anomaly hits, the model fails, and everything is wiped out. The model has only been wrong one year, but it cost you everything.

You don’t need a math degree to realize the importance of magnitude. “The frequency,” writes Nassim Taleb, in Antifragile, “i.e., how often someone is right is largely irrelevant in the real world, but alas, one needs to be a practitioner, not a talker, to figure it out. On paper the frequency of being right matters, but only on paper.”

But how are statistics- and historical-pattern-based models so wrong sometimes? Because the world is always changing. Consider the ancient advice of Heraclitus:

The river where you set your foot just now is gone: those waters giving way to this, now this.

Just because something worked in the past doesn’t mean it will work in the future. Even perfect historical models are just that — perfect at predicting the future only when it replicates the past.

If we are to have a hope of avoiding or at least minimizing the magnitude of the next crises, better models alone are not enough. We need to employ a cross-disciplinary approach. Let’s not just look at things through the lens of a purely economic model. Let’s borrow from psychology and engineering, too.

Engineers, for instance, deal with uncertainty by incorporating a margin of safety. Perhaps 99 percent of the vehicles that drive across a bridge are 10 tons or less. But there are still the infrequent trucks that exceed that. If we designed bridges the way we do economic policy, every once in awhile we’d drive 11-ton trucks over a bridge designed to carry 10 tons. That’s way too risky. Far better to design the bridge to carry 20 tons.

We must also re-examine incentives for people who work in finance. Our systems cannot be based on the premise, “Heads I win, tails you lose.” Upsides must be balanced with downsides. Warren Buffett seems to be onto something when he says:

If I were running things if a bank had to go to the government for help, the CEO and his wife would forfeit all their net worth. … And that would apply to any CEO that had been there in the previous two years. … I think you have to change the incentives. The incentives a few years ago were try and report higher quarterly earnings. It’s nice to have carrots, but you need sticks. The idea that some guy who’s worth $500 million leaves and only has $50 million left is not much of a stick as far as I’m concerned.

Buffett’s idea, of course that sounds a lot like like a modern manifestation of Hummurabi’s Code.

About bambooinnovator
Kee Koon Boon (“KB”) is the co-founder and director of HERO Investment Management which provides specialized fund management and investment advisory services to the ARCHEA Asia HERO Innovators Fund (, the only Asian SMID-cap tech-focused fund in the industry. KB is an internationally featured investor rooted in the principles of value investing for over a decade as a fund manager and analyst in the Asian capital markets who started his career at a boutique hedge fund in Singapore where he was with the firm since 2002 and was also part of the core investment committee in significantly outperforming the index in the 10-year-plus-old flagship Asian fund. He was also the portfolio manager for Asia-Pacific equities at Korea’s largest mutual fund company. Prior to setting up the H.E.R.O. Innovators Fund, KB was the Chief Investment Officer & CEO of a Singapore Registered Fund Management Company (RFMC) where he is responsible for listed Asian equity investments. KB had taught accounting at the Singapore Management University (SMU) as a faculty member and also pioneered the 15-week course on Accounting Fraud in Asia as an official module at SMU. KB remains grateful and honored to be invited by Singapore’s financial regulator Monetary Authority of Singapore (MAS) to present to their top management team about implementing a world’s first fact-based forward-looking fraud detection framework to bring about benefits for the capital markets in Singapore and for the public and investment community. KB also served the community in sharing his insights in writing articles about value investing and corporate governance in the media that include Business Times, Straits Times, Jakarta Post, Manual of Ideas, Investopedia, TedXWallStreet. He had also presented in top investment, banking and finance conferences in America, Italy, Sydney, Cape Town, HK, China. He has trained CEOs, entrepreneurs, CFOs, management executives in business strategy & business model innovation in Singapore, HK and China.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: