MACROECONOMIC FORECASTING: THE CAUSES OF FAILURES AND HOW TO CORRECT FOR THEM
Poor methods, bad models and inaccurate data are all blamed for the recurrence of serious forecast errors in econometric models used for policy. But surprisingly, these are not the primary cause of such systematic mistakes as the failures to forecast the inflationary surge of the late 1980s and the sharp rise in unemployment in the early 1990s. Rather, unanticipated large changes within the forecast period are the culprit. The primary fault in economic forecasting is not rapidly adjusting the forecasts once they go wrong. These are the conclusions of Professor David Hendry in his presidential address to the Royal Economic Society, published in the latest issue of the Economic Journal.

Hendry uses an analogy from rocket science: a rocket to the moon is forecast to reach there at a precise time and location, and usually does so. But if it is hit by a meteor and knocked off course (or destroyed), the forecast is systematically badly wrong. That outcome need not suggest poor engineering or bad forecasting models - and certainly does not suggest that Newtonian gravitation theory is incorrect!

Hendry’s approach to econometric modelling for policy purposes is to determine all sources of forecast errors, and to isolate those that cause systematic mistakes. This allows him to design methods to avoid them. While there are many possible explanations for forecast failures, he finds that it is purely due to substantial unexpected events in the forecast period (though the size of the failure depends on the model used).

Hendry argues that the primary fault in economic forecasting is not rapidly adjusting the forecasts once they go wrong. His research reveals methods for doing so - even if we don’t know the size, the sign, the location, or the timing of the shift. The resulting models are robust to systematic shocks after they have occurred, just as the rocket forecasts would be once the undesired course correction was discovered. The models are not yet robust to collisions with meteors!

The ability to correct bad errors shows that there are no implications from forecast failure to the policy relevance of a model. A bad model could avoid systematic errors; a good one for policy need not be as good at forecasting - NASA will probably use the same forecasting methods for its subsequent space shots. Econometric models have many parameters that might change and thereby lead to poor forecasts. Hendry shows that changes in means are the main source of systematic errors (as opposed to changes in such parameters as the speed of reaction, the variability, etc.).

Hendry notes that the most widely-used form of econometric model (due to his earlier research!) is not robust to mean shifts. Methods designed to offset such shifts are robust, but may not be useful for policy. As a result, the Treasury needs to find a balance. Even robust methods cannot avoid major forecast errors - to do so would involve predicting wars, earthquakes, meteors, changes of policy, etc. as well as knowing their consequences. But they are an insurance policy: they pay a little every period to avoid occasional big losses.

Hendry’s research opens a veritable Pandora's box. By allowing the economy to evolve and be subject to sudden large regime shifts (like the UK’s departure from the ERM in 1992), with models being possibly a poor approximation (especially after a shift), startling new theoretical results appear. For example, it cannot be shown that adding genuinely causal variables will improve forecasts; whereas adding a certain class of non-causal variables will certainly do so. Another important theoretical implication is that it cannot be determined in advance whether or not a given model will fail in practice.

Note: ‘The Econometrics of Macroeconomic Forecasting’ by David F. Hendry is published in the Autumn 1997 issue of the Economic Journal. Hendry is Leverhulme Personal Research Professor at Nuffield College, Oxford. His research was financially supported by the Economic and Social Research Council.

For Further Information: contact David Hendry on 01865-278554 (fax: 01865-278557; email: David.Hendry@nuffield.oxford.ac.uk) or RES/ESRC Media Consultant for Economics Romesh Vaitilingam on 0171-878-2919, 0117-983-9770 or mobile telephone 0468-661095 (email: rvaitilingam@cepr.org).