The unpredictable nature of renewable energy sources is generally seen as one of the aspects that make them hard to integrate in the energy system. In the incite blog of April 5th 2018, Jesus Lago has given an overview of different types and methods of forecasting. In this post, we dive deeper into current industry standards in renewables forecasting.
Day-ahead forecasts still central
Because day-ahead markets are the still the most important marketplaces where electricity is traded, the largest focus of the renewable forecasting industry is on day-ahead forecasts. For the Netherlands, where the day-ahead market closes at 1200h each day, this means that the forecasts have to be valid over a horizon of 12 to 36 hours ahead of time. Because traders need some time to determine their positions, most forecasts are usually issued around 900h in the morning. It takes a number of hours of computation time to actually produce the forecasts, so the initialisation time of a forecast that has to be issued at 0900h CET is at 0000h UTC. Hence, from the point of view of the numerical weather prediction (NWP) model, day-ahead forecasts cover the period of 24 to 48 hours ahead of real time.
The incredible skill of numerical weather prediction models
Despite the popular belief that weather models are always wrong, numerical weather prediction can be considered a huge scientific success. Take, for example, the day-ahead forecast (+24 to +48h ahead) of surface pressure and the actual realisations shown in figure 1 for the month January 2018. These forecasts are produced by the European Centre for Medium-range Weather Forecasting (ECMWF), which operate the world’s leading NWP model.
Figure 1 - ECMWF day-ahead forecasts and observations of surface pressure at Cabauw, The Netherlands.
Over a year, the error statistics, shown in figure 2, are impressive: a correlation coefficient of more than 0.99 and a RMSE of less than 1 hPa or 0.1% of the signal.
Figure 2 - Scatter plot of surface pressure forecasts against observations at Cabauw for the year 2018.
To be able to predict the large-scale pressure distribution with such an accuracy is nothing less than an amazing achievement. Interestingly, one cannot point out a single innovation that has led to the huge increase in skill of the weather models. It really is a series of continuous improvements and has therefore been called a ‘quiet revolution’ (Bauer et al, 2015).
Model output statistics
Pressure, however, is an atmospheric variable that is relatively smooth field, i.e. its variance on small spatial scales is small. Wind, on the other hand, is driven by pressure gradients, and shows much more variation on small spatial scales and is therefore more difficult to predict. Figure 3 shows the difference in spatial variation of surface pressure (left) and wind (right).
Figure 3 - Forecasted surface pressure (left) and 10m wind (right) over the Netherlands.
Because variables like wind, temperature and cloudiness (downward solar radiation) are much more difficult for weather models, there exists a large body of scientific literature dealing with the question how to improve weather model output using statistical post-processing techniques. These techniques are often referred to as model output statistics (MOS).
A simple MOS often has the following linear regression form:
where Y denotes, say, the wind speed at a wind farm, X1 is the forecasted wind speed of the nearest model grid point, X2 is the forecasted temperature, X3 is the cosine of the solar elevation, etc. In other words: a MOS model tries to correct for structural biases of a weather model by using the weather model’s own output variables. It could be, for example, that the weather model overestimates wind during cold nights. The regression model in the example above may correct for such a bias.
In the arena of commercial weather forecasting, companies try to distinguish themselves by developing good MOS models. One very effective trick that almost all commercial weather forecast providers use is the combination of different weather models. This works very well because of the statistical independence of the different weather models; after all they have been made by completely different institutes using their own models.
There is much ado about how the optimal combination of different forecasts can be produced. Fuzzy logic, random forests, simple regression techniques; anything has been tried. Ironically, it is often very hard to beat the simple average of different forecasts. This counterintuitive notion has been recognised in the scientific community for quite a long time (Menezes et al, 2000)
From wind to power
In the energy domain, another step is needed in the forecasting process: the conversion from meteorological variables to power. Owing to the growing importance of wind and solar power forecasts, a huge body of scientific literature has emerged in the past two decades or so about this topic (Kariniotakis, 2017).
Figure 4 - Scatter density plot of forecasted wind speed against power for a Dutch wind farm.
Figure 4 illustrates the challenge of producing a good wind power forecast. It shows observations of wind power of a Dutch wind farm against the forecasted wind speed for that location. If we recall that wind power scales with the cube of the wind speed (P ~ U3 ), we understand how small forecast errors in wind speed have serious consequences for the power forecast. For high wind speeds, power is limited to rated capacity of the turbines, but due to obvious economic reasons, this regime is not often reached (note the small number of points with a wind speed higher than 12 m/s in figure 4)
In the commercial forecasting industry, power forecasting is done using combinations of physical and statistical models. Aspects where physical models can add value are the modelling of local weather conditions and wake effects. Statistical models, on the other hand, are suited to deal with very short-term forecasts and to perform bias corrections and smoothing to minimize the errors.
Implications for the INCITE community
Despite the success story of numerical weather prediction, renewable energy forecasts remain prone to large errors and this will continue to be the case. Ironically, this is good news for the INCITE community, since smart energy systems can deal with these errors in various ways. We will have something to do in the next years! Flexible resources are a way to cope with unforeseen shortages in production or to mop up excess production. Model predictive control strategies can benefit from forecast updates, although one should not forget that even short-term forecast can have significant errors. In fact, we, researchers in the smart energy arena, should discipline ourselves and our students to model our systems in a stochastic framework, even when it involves pretty heavy mathematics. If we don’t and if we continue to model smart energy systems in a deterministic way, our research runs the risk of becoming increasingly meaningless. The pay-off will be that we can make a significant contribution to a better future by facilitating the integration of renewable energy sources. And, on a personal level, we may find deep rewards when we master optimization under uncertainty. After all, isn’t life itself riddled with uncertainty?
Bauer, P., Thorpe, A., & Brunet, G. (2015). The quiet revolution of numerical weather prediction. Nature, 525, 47.
Menezes, L. M. De, Bunn, D. W., & Taylor, J. W. (2000). Review of Guidelines for the Use of Combined Forecasts. European Journal of Operational Research, 120, 190–204.
Kariniotakis, G. et al. (2017). Renewable Energy Forecasting – From Models to Applications. Elsevier: Amsterdam.
