Understanding the superiority of multi-model ensemble forecasts through reservoir computing
TL;DR
It is shown that iterated forecasts over many time steps deviate from the ground truth along the unstable manifold of the target point, in both directions, so that, if forecast errors were independent and had zero mean, the arithmetic mean forecast should approach the true target like $1/\sqrt{\nens}$ where $\nens$ is the size of the multi-model ensemble.
Abstract
Weather forecasting and climate projection frequently use multi-model ensembles (MMEs) to improve short-term forecasts by averaging across models. However, this practice is often not well justified or validated. Using reservoir computing (RC) as a computationally efficient alternative to large-scale physical models, we assess the validity of the MME approach for chaotic time series. By training multiple randomly constructed RCs on the same dataset, we create a multi-model ensemble in which each model has its own unique error. These model errors lead to very different forecasting performances, with forecast error distributions that exhibit heavy tails. The arithmetic mean across forecasts from multiple models for the same target is usually closer to the ground truth than most individual forecasts, and further improvement is achieved by weighted arithmetic means where the weights are constructed based on each model's test-set performance. We show that iterated forecasts over many time steps deviate from the ground truth along the unstable manifold of the target point, in both directions, so that, if forecast errors were independent and had zero mean, the arithmetic mean forecast should approach the true target like $1/\sqrt{\nens}$ where $\nens$ is the size of the multi-model ensemble. We observe deviations from this behavior, which we attribute to the tails of the error distribution of random RCs.