Managing forecast risk
By John Stretch, Management education in finance and banking
ECONOMISTS, ACCOUNTANTS AND FORECASTS
“The Bank of England’s chief economist has admitted his profession misjudged the impact of the Brexit vote. Blaming the failure of economic models to cope with “irrational behaviour” in the modern era, the economist said the profession needed to adapt to regain the trust of the public and politicians. The bank has come under intense criticism for predicting a dramatic slowdown in the UK’s fortunes in the event of a vote for Brexit, only for the economy to bounce back strongly and remain one of the best performing in the developed world.
Official figures have shown the economy was outstripped only by the US among the large economies last year after growth in the third quarter was upgraded to 0.6%. Forecasts by the Treasury, the International Monetary Fund and the Paris-based OECD, all pointed to a recession after the vote, based on assumptions of steeply declining consumer spending and business investment.
He blamed the profession’s reliance on models that were built for an age when consumers and businesses, and especially banks, “behaved rationally”. Answering critics of the bank’s gloomy November forecast for the economy, he admitted that the bank did not anticipate the resilience of consumer spending after Britain voted to leave the EU. “
Source: The Guardian, Thursday 5 January 2017. (See Note 1)
As we enter 2017, many companies are busy finalising their annual budget. But how do you budget in a world where the best economists cannot predict the future? Perhaps the only approach is to prepare a budget as best you can, live with the uncertainty and then adapt to it and respond very fast when the future eventually happens.
This may explain why organisations have converted their annual budget process to a dynamic system of rolling forecasts. Traditional budgeting uses annual budgets – one fixed plan for the whole year - as the basis for target setting, planning and control. By contrast, rolling forecasts are continuous updates based on the best information currently available. Regular forecasting helps to quantify, and by implication manage, the gap between the original approved budget and the forecast. We, accountants, believe that forecast accuracy can be improved through analysis, learning, judgement, detailed supporting data and good systems. We analyse data to understand internal and external trends and are able to model a range of forecast options. We make plans so we have plans to change.
Rolling forecasts, we are told, facilitate event-driven planning and responsive resource allocation. The conventional wisdom tells us that regular forecasts can help us to understand and manage risk.
But how? There are many types of risk. In 2017 our countries, our businesses and we as individuals will be challenged to deal with a spectrum of risks that range from the daily risks of living that we take for granted, to risks we don’t know about and can’t even imagine. A single estimate of sales, costs and profits is based on choices and probabilities which are but one view of a broad range of possible futures.
In spite of this, it is my experience that many senior managers still want to work with one number, a single view of the budget or forecast (which they hope is reasonably accurate).
In this world of uncertainty, finance departments must invest time and effort to understand and classify risk, make risk explicit, and find ways to estimate the reliability of the forecast.
“Large, long-lived, and successful organizations may be more prone to misclassifying risk and uncertainty. Years of success can lead to a culture where conventional wisdom rules and leaders do not want to be questioned. If a risk is not understood, classified and debated, and if data and analytics are not explicitly part of decision-making, the organization may falter.
Businesses need to create a culture of healthy dissent, fact-based analysis and decision-making to identify all potential issues, implications, and actions. Unchanging strategies and tactics work until they don’t, sometimes with disastrous outcomes.” (See Note 2)
The South African grocery retailer Pick ‘n Pay reported declining profit margins and returns for 4 years in a row between 2010 and 2013, before remedial action was taken.
Prior to the invention of the digital camera, and for some years thereafter, Kodak didn't foresee the risks to their traditional business and eventually filed for bankruptcy in 2010.
Accountants need to communicate the risks in a budget or forecast to their managers.
It is, therefore, critical that any budget and forecast should be supported by a detailed commentary on the risks implicit in the numbers, in four areas:
- the estimated accuracy of the forecast and its upper and lower limits
- explanation and classification of the relevant downside and upside risks and uncertainties.
- trends in technology, products, customer wants, and monetary trends in sales volumes and prices, raw material prices and other input costs
- alternative outcomes and scenarios
THE ESTIMATED ACCURACY OF THE FORECAST AND ITS UPPER AND LOWER LIMITS
Forecast accuracy is unlikely to be consistent throughout the forecast. Thus a single point figure of the reliability of the forecast (“this forecast is 90% reliable”, for example), is misleading. An estimate of reliability should be provided for each phase or time period of the forecast.
Project managers describe a learning process known as the “cone of uncertainty”. At the beginning of a project, comparatively little is known about the project, and so forecasts are subject to large uncertainty. Uncertainty decreases over time. As more is learned about the project, the uncertainty tends to decrease, reaching zero at the end of the project.
Cone of uncertainty. Source: Wikipedia
Similarly, forecast uncertainty increases as we move further into the future. We are usually able to attach greater confidence to the forecast for the month or quarter immediately ahead, than for the month or quarter 12 months ahead.
The forecast period within the “cone of certainty” will influence the method of forecasting and the information which is used. Refer Appendix 1 for an industry-specific example.
DOWNSIDE AND UPSIDE RISKS AND UNCERTAINTIES
The American politician Donald Rumsfeld was once asked: “Does the government of Iraq have weapons of mass destruction which it can supply to terrorist groups?”
He replied “There are known unknowns. That is to say, there are things that we know we don't know. But there are also unknown unknowns. There are things we don't know we don't know. And if one looks throughout the history of our country and other free countries, it is the latter category that tends to be the difficult ones.”
Donald Rumsfeld 2002. Source: Wikipedia
(See Note 3)
“Known unknowns” refer to risks you are aware of, such as a major contract being cancelled. We try to understand these by identifying the upper and lower limits of each item, the probability of occurrence, and the past and likely future frequency.
Unknown unknowns are risks that come from situations not thought of, or have not been considered possible, or are so unlikely that they have never been considered worth forecasting. Mathematician Nassim Taleb uses the term “black swans” to describe these rare unplanned events with severe impact that, with hindsight, were predictable or explainable. Prime political examples in 2016 were Brexit and the Trump presidency. Black swans remind us that there are limits to knowledge when it is based only on observations and past experience. (See Note 4)
The “known unknowns” and how they have been dealt with in the forecast must be made explicit in the commentary.
Accountants and economists use trend data to try to uncover and convert uncertainty into known risk. We believe that “the trend is your friend.” (See Note 5). We use our understanding of trends to project the past into the future. For example, we can usually justify our assumptions about demographic shifts and certain new technologies.
Trends in technology, products, and customers, should be identified with an explanation of how they have been dealt with in the forecast.
Similarly, monetary trends in sales volumes and prices, raw material prices and other input costs should be highlighted.
Some trends such as technological change and new product sales may be exponential rather than linear and thus more difficult to forecast.
ALTERNATIVE OUTCOMES AND SCENARIOS
A single figure forecast reflects the preparer’s view of a likely outcome, based on choices from a multitude of alternatives. The forecast presentation should include a discussion of alternative outcomes and scenarios.
In scenario planning, we combine assumptions, trends, knowns and unknowns into a limited number of permutations that speculate on alternative views of the future.
We make assumptions about factors that will affect the nature of the future world in which the organisation operates. Examples are demographic shifts, new technologies, future interest rates, outcomes of political elections, rates of innovation, fads and fashions in markets.
These factors are analysed to understand how these major drivers for change will impact on key assumptions about the future. These are used to generate a wide range of scenarios which are then reduced to 2 or 3 most likely scenarios which are evaluated in detail.
In an uncertain and fast-changing world, line managers need to be made aware of the uncertainties and risk inherent in the financial forecasts provided to them. Uncertainty is difficult to manage but uncertainties can be converted into known risk as forecasting capabilities and data management improve. When risk is understood, it can be categorised, mitigated, managed, hedged or even avoided. Uncertainties require continual review to identify changing facts and circumstances that affect risk. In the future, winning organizations will have a greater ability to identify, understand and manage risk.
Note 1: A question of timing?
Answering critics of the bank’s gloomy November forecast for the economy, their Chief Economist admitted that the bank did not anticipate the resilience of consumer spending after Britain voted to leave the EU. But he said that he thought the bank was wrong about timing not about the fundamentals, and that the Bank of England still expected Brexit to harm growth.
Note 2: Reference
This quote is from an excellent article “Big Data: Blurring risk and uncertainty” by Bill Pieroni, chief executive officer of Accord, featured in LinkedIn Technology, August 22, 2013.
Note 3: The “unknown knowns”.
Beyond these three categories, there is a (psycho) logical fourth, the “unknown known”, where people are in denial about (or intentionally refuse to acknowledge) facts and issues they subconsciously know to be true. Rumsfeld thought that the main dangers in the confrontation with Iraq were the 'unknown unknowns', that is, the threats from Saddam which the United States suspected but could not prove. But often the main risks lie in the ‘unknown knowns’ - the moral values, beliefs, suppositions and practices people disavow and pretend not to know about. Did the US know there were no weapons of mass destruction in Iraq?
Note 4: Adapting to “black swans”
A colleague in the Reinsurance Industry suggests that many of these so-called once-in-a-lifetime, uncertain events actually occur with regular frequency and potentially knowable impact. Some Black Swan events occur every few years (earthquakes, tsunamis, stock exchange crashes) while others are separated by longer, yet regular, time intervals. The regularity of these events suggests that black swans are giving way to shades-of-grey swans. The Reinsurance Industry continuously monitors statistical data to identify, model, and mitigate black swan risks.
Note 5: The trend is your friend
“Extrapolating from the past is like driving down the road looking in the rear view mirror... fine until you come to a bend in the road” (Toby Wilson, Microsoft Financial Director, UK.)
Appendix 1: Suggested sources of information for rolling forecasts in different time periods