wave background

All forecasts are wrong: tips for improving accuracy

Forecasting

forecastsAll forecasts are wrong. This infamous statement may be among the first things professors preach in many university-level business courses on forecasting. Most of us are familiar with the frustration felt when we are unable to close a project within the budgeted scope, year-end business volumes are nowhere near the January forecasts, summer vacations are ruined by rain appearing out of nowhere and savings yield little profit in the stock market.

With forecasting so prone to error, why then do we continue to rely so much on them for critical decision-making? A reasonable hypothesis is that to be able to start planning the future, we feel that our planning efforts need to have a basis. It would be fair to say that most decision-makers are aware that predicting the future with 100-percent accuracy is impossible. Widely accepted is the fact that forecasting contains some margin of error and that forecasts with a lower margin of error are the ones that make a difference.

A healthier approach would therefore be to stop questioning why forecasts are always off and instead, try to minimize the margin of error. After all, in a world where chimpanzees beat Wall Street professionals in competitive salaries, room for improvement must be possible.

How then to maximize the accuracy of your forecasting? Here are some useful tips:

Know and embrace your errors
No one is Nostradamus. Errors are an inherent part of your forecasting. So, minimizing the margin of error while staying loyal to your overall data quality – making assumptions, rounding up or down statistics, etc. – will determine the success of your error handling. You may recall high-school science experiments when using one margin of error for one variable that was in direct correlation to another, it was useful to stick to the same margin of error with inter-dependent variables.

Another useful and often overlooked rule of thumb is to remember that the further we try to predict in the future, the greater normally the margin of error. Think weather forecasts: tomorrow’s forecast is often more accurate than that of next week’s.

Improve the quality of your data
Gathering, refining and analyzing data for forecasting is an art in itself. In this day and age with the use of big data skyrocketing, more and more data can be tracked, stored and crunched in a multitude of ways. Compared to forecasting in the 70s and 80s, finding the most relevant data – not just any data – has never been more important and is a challenge that forecasting professionals face today.

A good way to increase your relevant data sampling is through crowdsourcing. Sports Interactive (SI), the makers of the digital game, Football Manager, recently released a B2B version to professional football clubs, which successfully used and popularized the crowdsourcing model. Contrary to the traditional scouting model of sending individual scouts to different national leagues, SI’s crowdsourcing network of over 1,300 scouts (who constantly get feedback from fans and Football Manager players) – give much more updated and reliable data on individual players and eliminates the subjective views of individual scouts.

If you’re unable to outsource your data gathering in any way, a similar strategy could be to outsource your data analysis: i.e. crowdsource the forecasting process to different individuals in an organization rather than just delegating it to a few people. I’ll discuss this crowdsourcing aspect at length in a later blog.

Forecast more often
If you have to forecast, forecast often.” This quote should be a no-brainer. The more frequently you forecast, the more you narrow down your forecasting horizon. This, in turn, allows you to do a reality check and reduce your margin of error.

Train and/or recruit “super forecasters”
The herd instinct among forecasters makes sheep look like independent thinkers” (Edgar R. Fiedler in The Three Rs of Economic Forecasting-Irrational, Irrelevant and Irreverent, June 1977)

Super forecasters are not aliens or uniquely smart people. The talent to forecast can be taught, nurtured or, if training existing personnel is unaffordable, recruited.

In a recent forecasting blockbuster book, Superforecasting: The Art and Science of Prediction, author Philip Tetlock states that most super forecasters have above-average IQs (not necessarily geniuses), strong statistical mindsets and the courage to go against pre-determined decisions or prejudices, that is, the power to say no and present their findings despite top management expecting them to forecast a particular scenario that supports their own arguments.

Gamification – i.e. arranging forecasting tournaments and competitions – is key to crowdsourcing the forecasting effort, increasing its accuracy as well as motivating and developing super forecasters.

The right software
You’ve improved the quality of your data, become more aware of the margin of error, decided on the optimal forecasting frequency and trained/recruited the right personnel. What’s the missing link in the chain? It’s investing in the right software, which, like the Big Lebowski’s rug, is the element that ties the room together. Briefly, the four main characteristics of good forecasting software are as follows:

  • It uses the right language and algorithm (contingent on the scientific expertise of the software provider)
  • It is stable for continuous use
  • It provides a good user interface – critical for training personnel
  • It offers good reporting tools (enabling visualizing findings for top management)

Inherent randomness
You may have done everything right but you must still embrace the inherent randomness of every forecast. Sure, the level of randomness differs from forecast to forecast but accept the fact that luck – i.e. force majeure events – plays an important role in every forecast. It is therefore essential to prepare for the worst.

To be continued in the next blog posting on forecasting.

2 comments

  1. Hello,

    I enjoyed reading your post because I am very concerned about minimizing the margin of error in my forecasts for our contact center. Would you mind answering a few quick questions? You mentioned a book, “Superforecasting: The Art and Science of Prediction”, would that have any practical use for a WFM Analyst? Can you recommend any other books or materials that would be useful? I recently started going back to school; in your education what courses did you find beneficial in the area of WFM and forecasting? I have heard from some people that economics is a good subject for this field, would you agree?

    Thank you

    1. Hi Bruce,

      Thank you for the question.

      Tetlock’s book Superforecasting is not a WFM-specific book by any means. I would say that the book is in the intersection domain of forecasting, popular science and readable economics literature hence it has become a bestseller in its domain. This being said, I thoroughly enjoyed reading it and am coming up with new ideas related not only to WFM forecasting but even business processes and general business planning every week since having read the book. I would say that it gives the reader a very solid foundation of forecasting basics in an introductory fashion. Therefore I would recommend it to anyone.

      Other literature I would recommend would be Kahneman’s “Thinking: Fast and Slow” (a more advanced version of “Superforecasting”), and “Call Center Staffing” by Penny Reynolds.

      In terms of courses, I would recommend any entrance-level forecasting course, statistics and queuing theory courses. Most of the big players in today’s WFM planning software industry are using Erlang C as their main programming language when it comes to forecasting incoming call workloads. Therefore more advanced courses digging deeper in Erlang, Poisson and other queuing-related distribution techniques could also help. I’d say that in overall, having a solid foundation of queuing theory never disappoints in the WFM branch. Although some of these courses could also be found in curricula of various economics programs, in my experience (i have an undergrad in Operational Research and a master’s in Management) it was mostly the engineering classes that helped me when it comes to specifically WFM.

      One thing I would also like to highlight is that forecasting is only the 1st part of the WFM puzzle. When it comes to scheduling then it’s all about scheduling algorithms, optimization etc. And that is a whole different domain on its own.

Leave a Reply

Your email address will not be published. Required fields are marked *

Contact me I want to know more