C3S energy: an operational service to deliver power demand and supply for different electricity sources, time and spatial scales over Europe

The EU Copernicus Climate Change Service (C3S) has produced an operational climate service, called C3S Energy, designed to enable the energy industry and policy makers to assess the impacts of climate variability and climate change on the energy sector in Europe. The C3S Energy service covers different time horizons, for the past forty years and the future. It provides time series of electricity demand and supply from wind, solar photovoltaic and hydro power, and can be used for recent trends analysis, seasonal outlooks or the assessment of climate change impacts on energy mixes in the long-term. This paper introduces this dataset, with a focus on the design and validation of the energy conversion models, based on ENTSO-E energy data and the ERA5 climate reanalysis. Flexibility and coherence across all countries have been privileged upon models’ accuracy. However, the comparison with ENTSO-E data shows that the models provide plausible energy indicators and, in particular, allow to compare climate variability effects on power demand and generation in an homogenous approach all over Europe. This preprint is ©2021, RTE. This article is not under review at any journal.


Introduction
The power sector is exposed to weather and climate variability at all timescales, with impacts on both demand and supply [1]. In the next decades, major changes in the sector will be necessary to achieve the energy transition required by the Paris Agreement [2]. Low carbon electricity generation will play a key role in the path towards net-zero CO2 emissions, as ambitioned by the European Green Deal [3].
The increasing share of renewable generation, mainly from wind and solar energy together with hydro power, will increase the exposure of the energy sector to climate variability and change [1,4]. In the past few years, several studies have looked at the feasibility of a European power system with large shares of renewable generation [5][6][7][8][9]. Similar studies have been carried out for other locations [10][11][12][13][14][15]. These studies mainly deal with different scenarios of electricity mixes to evaluate their technical and economical feasibility, in particular to assess flexibility and reserves needs. However, climate variability and climate change impacts are either not addressed or only partially. [7] for instance used only seven years of climate data, which is too short to capture year-to-year climate variability. Silva et al. [5] used 31 years of wind and solar generation, reconstructed on the recent past from reanalysis data (from ERA-Interim), which to our knowledge is the most comprehensive approach at the time of this study. Climate change impacts on hydro, wind and solar power generation have been studied by several authors [16][17][18][19] but currently, these studies do not provide open and easily accessible datasets which can be used by energy practitioners and decision makers to run energy mixes studies, via energy system modelling. Running European power system studies requires energy datasets which are coherent, using homogenized and long-term climatic data sources, to be able to properly link energy demand and production to climate variability and impacts of future changes. Moreover, this needs to be done at a continental level in order to investigate possible im/balancing effects across a wide area (Europe in this case).
In the past few years, several datasets have been developed with this goal. The EMHIRES [20,21] and Renewables.Ninja [22] datasets in particular, provide time series of renewables capacity factor for European countries covering the last three decades, but they focus only on wind and/or solar generation, whereas hydro power has been addressed only recently by the Joint Research Centre 1 and no data is available for electricity demand in the extent described here. In addition, these databases do not integrate the expected impact of climate change on the renewable energy sources (RES) power generation time series. Currently available datasets to run European power system studies therefore present some limitations, for instance too short datasets, or no homogeneous approaches between demand and supply from various sources, missing variables (e.g. demand or hydro), or no future projections available. This leads to a situation where RES power generation time series are calculated independently in each prospective study. In addition to the seemingly unnecessary duplication of effort, having disparate ad hoc approaches to producing input energy data makes it difficult to compare the outcomes of such studies. There is thus a need for a unified dataset including all relevant energy variables and time scales, to allow enough flexibility to easily integrate them in any prospective power system analysis.
The Copernicus Climate Change Service (C3S -https://climate.copernicus.eu/) was launched in 2015 to lead and coordinate development of climate service infrastructure and underlying data provision mainly at the European level. The Sectoral Information System (SIS) component of C3S makes use of the C3S climate data to meet the requirements of users, with a specific focus on purveyors and policy makers.
The SIS includes various sectors of which energy is a prime example. The datasets produced by the EU C3S Energy operational service (C3S-E) precisely address the above stated needs. The description of these datasets and their development is the objective of this paper. The result is a coherent and unified dataset that can be illustrated by the data cube in Figure 1. Figure 1 : The C3S Energy operational service data cube representing the availability of energy datasets for each source (physical dimension), climate streams (temporal dimension) and geographical aggregation (spatial dimension). Note that although all these elements are available from the C3S Energy operational service, in this paper we present only the historical stream.
The C3S-E dataset brings together electricity demand, with wind, solar and hydropower generation (physical dimension), for the three climatic streams, historical (from 1979 to present, with a possible extension back to 1950 anticipated in the near future), seasonal forecasts (from present to 6 months ahead) and projections (out to 2100). A range of spatial resolutions is available (geographical scale): at a grid resolution of 25 km, but also as subnational (NUTS2 2 ) and national (NUTS0) aggregates [23].
Additionnally, national (MAR0) and regional (MAR1) maritime regions have been defined for offshore wind energy [24], because no such aggregation regions exist in the Eurostat reference. Overall, the C3S-E dataset is designed to fulfil the needs of both potential users, who want to assess the impact of climate on energy operations, management and planning, and energy modellers, who need a user friendly dataset ready to be utilised for their power assessment studies. The C3S-E dataset builds on the previous C3S dataset, produced by the European Climatic Energy Mixes (ECEM) project [25]. With C3S-E both the underlying climate data and the energy models have been updated. In brief, the C3S-E dataset is based on state-of-the-art climate data covering the three streams: historical, seasonal forecasts and projections.
It is important to acknowledge that the dataset is not designed for operational activities, because the energy models are simplified compared to operational and more accurate models. Its objective is to illustrate the impacts of climate variability and climate change on the European power system, through the effects on demand and renewables generation. This paper focuses on the energy conversion models used for building the C3S-E dataset. These models have been developed and calibrated for the historical stream using the ERA5 reanalysis and then applied to all climatic data. Evaluations of the seasonal forecasts and projections streams are not discussed here.
The different data sources used by the C3S-E dataset are described in section 2. The models chosen to simulate electricity demand and supply as well as the motivations for these choices are detailed in section 3. A validation of the selected energy models over the historical stream is presented in section 4.
Strengths and weaknesses of our approach as well as possible improvements and extensions are finally discussed in the concluding section 5.

Climate data
The climate data for the C3S-E historical stream come from the European Centre for Medium-Range Weather Forecasts (ECMWF) ERA5 reanalysis [26]. Reanalyses are gridded datasets covering the globe, as in the case of ERA5, or a regional domain. They are reconstructions of past climate produced through the assimilation of observations in physical numerical models which have been developed explicitly for climate monitoring and research.
Unlike in the case of the ECEM dataset [25] which was based on the previous ECMWF reanalysis, namely ERA-Interim, and for which bias adjustment was required [27], with ERA5 the need for bias adjustment was not considered as critical, given its generally superior quality [26]. Therefore, to develop the historical stream of the C3S-E energy dataset, ERA5 is used in its original form. The only data manipulation comes from interpolation onto a regular 0.25 degree grid as well as the country averaging (NUTS0).
The specific climate variables used to develop the C3S-E models are the following: • air temperature (at 2 m height) • precipitation • downward solar surface radiation (also known as Global Horizontal Irradiance) • wind speed (at both 10 m and 100 m heights) Their temporal resolution is one hour, with the distinction that for air temperature and wind speed the values are instantaneous (at the hour) whereas for precipitation and solar radiation they are accumulated (over the previous hour).

Energy data
Being part of the Copernicus services, C3S-E aims at providing free and open access data, including for energy indicators. One of the major constraints in developing and assessing the C3S-E models was to identify and use freely available data, in order to fulfil this open access policy. The other constraints were: • Availability of data for all European countries; • Physical relevance both in space and time among the different variables (demand and supply from wind, solar and hydro); • Sufficient temporal data coverage to set up and train models, and then validate these on an independent period.
Even if multiple energy datasets exist, only few of them fulfil the stated requirements. The choice was then made to use mainly the European Network of Transmission System Operators 3 for Electricity (ENTSO-E) databases. The ENTSO-E is an association of 43 electricity transmission system operators (TSOs) from 36 countries across Europe, established by the EU in 2009.
In this work we have used mainly the demand and generation data provided by ENTSO-E in two different forms: the Transparency Platform 4 (ENTSO-E TP hereafter) and the Power Statistics (ENTSO-E PS). From January 2015, the ENTSO-E TP made all the power grids data for all the European countries available according to the rules imposed by the EU regulation 543/2013 5 . The platform was created to provide high-quality and timely available data to the energy markets' participants. It includes supply and demand data for all the European countries in a homogeneous and easy to access way. A review of the ENTSO-E TP, the methodology used to create the datasets and issues and limitation can be found in [28] or [29]. ENTSO-E TP data has been used for generation. For demand (also called load), we used the ENTSO-E PS 6 "Monthly Hourly Load Values" dataset, which goes back to 2006 for some countries, and then provides a longer record length for models calibration. This dataset, which has been discontinued in 2019, also contains more robust and validated data than ENTSO-E TP, that has doubious data especially for the year 2015.
The type of generation data that the TSOs calculate and provide on the ENTSO-E TP is described in the EU Regulation 543/2013 (for example in Articles 14 and 16). This means that some differences can appear when comparing aggregated data from the ENTSO-E TP with similar data obtained from other data sources. In particular, the installed capacity data used in C3S-E is obtained from the dataset 14.1.A (Installed Generation Capacity aggregated) and consists of the sum of installed capacity for all the units with at least 1 MW of capacity in a specific area. As described in [28] and [29], ENTSO-E data have some issues and inconsistencies. The Open Power System Data platform 7 is a very good source of information. It provides easy access to ENTSO-E and other sources of data, after some validation and cleaning, together with pieces of code, in an open and free access approach. It has been used here to run some analysis of ENTSO-E data from the TP. Figure 2 shows a visual summary of all the countries and energy variables modelled in the C3S-E dataset. In this paper and the corresponding dataset, countries are identified with their ISO 3166-1 alpha-2 code. More details about the availability of each data set are given in section 3.

Description of the energy conversion models
The main goal of C3S-E is to allow users to model and assess the effects of climate variability and climate change on electricity consumption and generation. Therefore, the assumption is made that the energy system is constant in terms of installed capacity and load profile. This means that the generation 6 https://www.entsoe.eu/data/power-stats/ 7 https://open-power-system-data.org/publications capacity is fixed in time, and the long-term trend in energy demand due to, e.g., economic growth or population change, is removed. Then, once the models are set up, one can easily re-introduce generation capacity evolution or energy consumption trends by adding these external factors to the modelled climate-dependent part of demand and supply. The models described here have been set up using the ERA5 climate reanalysis for climate variables and ENTSO-E data from 2006 onwards for demand, and from 2015 onwards for generation. Table 1 shows the climate variables which have been used to compute each energy indicator, with the following abbreviations: • EDM: Electricity DeMand; • WON: Wind ONshore; • WOF: Wind OFfshore; • SPV: Solar PV; • HRE: Hydropower generation from REservoirs; • HRO: Hydropower generation from Run-Of-river. It has to be noted that, in general, installed capacity data are available at a yearly time resolution (in particular in the ENTSO-E TP). Computing capacity factor at a daily or hourly time resolution then requires to interpolate the yearly values to daily/hourly values. This process of course introduces some inaccuracy as, in real world conditions, the installed capacity does not grow linearly over time, but depends on the date of commitment of each generation unit. This approximation then introduces some uncertainty in the actual values of capacity factors, and this needs to be taken into account in the models validation process.
To summarise, the models used to generate the C3S-E data are the following: • Electricity demand (EDM): Generalise Additive Model (GAM) described in Section 3.1; • Hydro power (HRO and HRE): Random Forest (RF) model, described in detail in [30] and summarised in Section 3.2; • Solar power (SPV): a physical model, described in detail in [31] and summarised in Section 3.3; • Wind power (WOF and WOF): a basic, standard model described in Section 3.4.

Electricity Demand models
Generalized Additive Models (GAM) were chosen as the preferred approach to model daily electricity demand. GAMs are a generalization of linear models but can embed non-linear (analytical) functions to capture the relationship between the predictors and the target variables. Introduced by [32], they have been improved by [33] and many others since then. They are well known methods for load forecasting [34][35][36], and have been successfully used in the GEFCOM2012 forecasting competition [37]. Their major advantages are that they are easy to interpret, fast to run and adaptable to different datasets. This approach requires observed load data for both the training and validation periods. The data used here are from the ENTSO-E Power Statistics (PS) database, namely the "Monthly Hourly Load Values" described in section 2.2. The demand models have been developed at country (NUTS0) level and daily time resolution. However, the ENTSO-E demand data is available at hourly resolution, and future improvement to the models developed here could take advantage of the higher time resolution, which is useful for adequacy (supply-demand balance) simulations. One GAM has been built for each country of The selection of predictors varies from country to country and even from season to season, but they basically include different terms linked to: The climate variables can also be combined. For instance, a term can be added to consider GHI only in winter days. In addition, one or several smoothed temperatures over a few days can be considered, to account for the delayed effect of outside air temperature on electricity demand, mainly because of buildings' inertia. The choice to use combined variables and smoothed temperatures is made by iteration, in order to minimize the residuals of the model. All models have been developed with the R software.
The modelling approach is the same for all the 32 countries considered, as described below. Only the start and end dates of the training and validation periods differ, based on ENTSO-E data availability and quality. The process consists of 4 steps, as described in Figure 3 for the case of France:  At the end of the process, the GAM parameters of each country are stored as R object data, which can then simply be loaded to be reused with any given climate dataset, including seasonal forecasts and climate projections.

Hydro power generation model
Modelling hydro power generation at a pan European scale for all countries is a challenging task. It would in principle require using an extensive amount of information, for example: river flow data measured at the inlet of the plant (reservoir or turbine), hydro power plants technological characteristics, management scheme of the power plants (e.g. if it is used for balancing, for seasonal storage, and if it is part of a set of multiple plants in the same basin). Moreover, to calibrate and assess the quality of any approach, operational data should be used for validation purposes (e.g. measured generation timeseries). Unfortunately, such information is impossible to obtain consistently for all the European countries.
The approach we use has been published in detail in [30]. It provides a dataset of reservoirs (HRE) and run-of-river (HRO) hydro power generation, aggregated at country level and daily time resolution, for Slovakia (SK) for HRE; for HRO, the list is the same except Sweden but with Finland (FI) (see Figure   2).
The methodology is based on the random forests (RF) machine learning approach, and uses country aggregated predictors from ERA5, namely air temperature (TA) and total precipitation (TP) (see Table   1). In addition to the TA and TP on the same day as the simulated generation, the model uses time lagged TA and TP of up to 200 days.

Solar power generation model
Classical approaches to the calculation of the photovoltaic (PV) power generated in a region from meteorological data require the knowledge of the detailed characteristics of the plants, which are most often not publicly available. Our approach follows the objective to obtain the best possible assessment of power generated in any region without having to collect detailed information on PV plants. It is based on a model of PV plant coupled with a statistical distribution of the prominent characteristics of the configuration. A short description of the modelling approach as well as a description of the approach used for the assessment of the reference configuration and their weights are provided here, since the algorithm has already been published in [31].
The model is based on the simple idea that the aggregated PV power generated in a region is the sum of the normalized outputs of all plants with characteristics ! ! multiplied by the proportion " ! of plants having the characteristics ! ! in the whole set of plants installed in the considered region. The regional PV power generation can therefore be expressed as follows: The reference configurations have been chosen on the basis of the statistical analysis of ca. 30,000 PV installations in Germany. It was chosen to use 13 reference configurations which is a compromise between a limited number of configurations and the best possible modelling accuracy in a tractable approach. As detailed in [38], the weights have been derived from the above-mentioned statistical analysis and by a simple geography dependant parameterization allowing to then generalize the characteristics of the German installation to any region in Europe.
The calculation method for SPV requiring numerous computations, it is hardly tractable for large datasets like multi-model multi-emission scenarios climate projections or operational multi model seasonal forecasts. This high number of calculation is needed to properly take the variation of the sun position into account, but it is not necessary for the coarser resolution of seasonal forecasts and climate projections. As a result, a simple parametrization of the model has been developed to speed up its implementation without loss of accuracy at daily time resolution for the seasonal forecasts and projections streams of C3S-E (see supplementary material).

Wind power generation models
Our approach assumes a single wind turbine type, with a fixed hub height, homogeneously located on a regular grid, to overcome the lack of data and complexity needed to run a wind power physical model. Similar approaches has been used in many studies like e.g. [16]. One of the advantages is that it does not require any assumption or data relative to the exact location of wind turbines. Two different wind turbine models are considered, for onshore and offshore respectively, based on the actual trend in wind turbine installation and expert advice: It is assumed that wind turbines are distributed uniformly on all grid points, which obviously is not the case in the real conditions. This simple approach of course is not intended to reproduce accurately the actual capacity factor as reported for instance by ENTSO-E, also because of the simplification of using a single wind turbine type and hub height.
In addition, this approach is not able to take into account maintenance periods and various losses (due to wind turbines conversion systems, electrical losses, wake effects…). However, it is a very easy way to reproduce the variability of the capacity factor based on wind speed variability. Hence, the capacity factor values are not supposed to be close to actual values, but the variability is in principle well reproduced, as can be measured by correlation coefficients.

Validation of the C3S-E energy conversion models
This dataset as built to fulfill a Copernicus Climate Change Servicec contract. One of the key requirements was that the energy models were designed to be applied in any European region and with different kinds of climatic data (reanalysis, seasonal forecasts and projections). The most important criteria for the model selection are therefore those related to the flexibility of the models and the consistency and coherency among variables, both in space and time. Given that there are many regions in Europe with limited information on the modelled energy sources, the most accurate individual models that require very specific information could not be selected. Flexibility was therefore given more consideration than absolute accuracy in the model selection. The validation presented here is therefore aimed at validating the plausibility of the models output rather than demonstrating that the C3S-E approach outperforms individual models from the literature. This work is a first step in providing energy data for the different variables, for all EU countries and for the multiple timescales addressed in C3S-E.
Possible and planned improvements to the models are discussed in section 5.

Reference dataset
As described in section 2.2, the ENTSO-E datasets (PS and TP) are, to our knowledge, the only source of homogenous data available for all European countries. These datasets, while providing a very good reference for C3S purposes, nonetheless present some drawbacks as explained for instance in [28] and [29]. The most problematic issues for C3S-E are the following: i) Record length and quality differ among countries. There is no easily accessible documentation on the reporting and processing methodologies applied to the data, which may change over the years, creating inconsistencies. Discontinuities that can be only explained by a change in the processing method have been identified for a few countries, and some countries have a non negligible ratio of missing or obvious erroneous data. In addition, the PS dataset was discontinued in November 2019, and the only data available for demand then comes from the TP. Some inconsistencies between PS and TP have been found too (not shown here), which for instance create offsets between the two datasets.
ii) In the ENTSO-E TP, inconsistencies are found between different datasets, in particular for the first years. In many cases, the actual generation is not consistent with the installed capacity (e.g. the solar production of PV in Italy).
The installed capacity from ENTSO-E TP indeed shows strong deviations from alternative data sources for several countries. Figure 5 shows for instance the installed capacity for solar power from four sources  Table S1). Accordingly, the validation metrics and figures shown in the next sections are most likely to be relevant for this year and need to be interpreted with caution considering the installed capacity data issues for the other years. A way to overcome this uncertainty around the installed capacity values is to represent C3S-E capacity factors (for wind and solar energy) against ENTSO-E TP generation data (see the scatter plots in the next sections).

Figure 5: Installed capacity for Solar Power in selected European countries from diffferent sources. ENTSO-E PS refers to Power Statistics, ENTSO-E TP is the Transparency platform, the purple line for France comes from RTE, the French TSO.
The above-mentioned issues are unavoidable considering the complexity of energy data regulation as well as technical constraints related to the reporting of the RES installations, especially in periods marked by a strong increase of the installed capacity. Although data must be used with caution, it should be noted that the situation is improving constantly, driven by i) efforts of TSOs and ENTSO-E to improve data sharing and data quality, and ii) the feedbacks made by users in return for the public access policy of these data.
In summary, even if the ENTSO-E databases present some issues, they constitute a good framework for setting up and validating the models used here. It has however to be kept in mind that the abovementioned issues have to be considered as an additional source of uncertainty in the modelling process.

Validation results
In this section, we use various metrics to evaluate and compare the quality of the modelled variables.
The Pearson correlation coefficient is the most used measure in this work for the following reasons: 1. It is well-known and widely used; 2. It is scale-independent and thus very useful in comparing variables with different scales (e.g. capacity factor or power output) and is thereby less sensitive to error on the installed capacity; 3. It measures the covariance of two variables, so it is particularly suitable to assess the capability of the proposed models to reconstruct and capture the variability of the energy variables, one of the aims of the C3S-E dataset.
We have also applied the Mean Absolute Error (MAE) to describe the error using the same unit of measure as the energy variable. Finally, when possible, we have also used relative measures such as the A quick overview of the performance of the different models is given by Figure 6 which displays the correlation coefficient computed on the validation period as described above (the correlation coefficient values can be found in Supplementary material Table S2, and Table S3 for demand). It is important to show the performance of a model with respect to the average generation (thus the installed capacity) because there is a clear link between performances and amount of installed capacity measured by the ENTSO-E dataset. Figure 6 shows that, overall, the correlation coefficient between the reconstructed variables and the ENTSO-E data is good. For demand, Luxembourg is an exception, due to issues in the data available to train the model. Hydropower shows better results for run-of-river (HRO) than reservoirs based (HRE). This is expected as HRO generation is more directly driven by climate variability than HRE, which embeds the management of reservoirs in supply-demand balance operations.
A general rule is that the models perform better when the installed capacity is bigger. This comes mainly from the fact that low installed capacity corresponds to a low number of plants in the country, and then less smoothing of errors when averaging at country level. We will discuss more in detail the performances for each variable later in this section. Table S2 for generation and Table S3 for demand.

Figure 6: Overview of correlation coefficient of the C3S-E data compared to ENTSO-E, as a function of the average load/generation from the ENTSO-E dataset. Correlation coefficient numbers are given in Supplementary Material
Displaying aggregated metrics is not enough to assess the quality of the provided modelled outputs, to this end we show in Figure 7 the scatter plots for three selected countries at the maximum available temporal resolution (daily for hydropower and demand and hourly for wind and solar). Figure 7 shows three main aspects of all models: i) an overall good fit with the observations, considering the simplicity of the models; ii) a significant dispersion in the results and iii) some issues, especially for wind power, in particular "flat" capacity factors of around 0.1 whereas actual generation varies in a large range . The low correlation coefficient in the modelling of hydropower reservoir in Germany (DE) is due to the low installed capacity.
The issues reflect both the simplifications made in the models, but also the lack of quality of some data in the ENTSO-E database. More details are given in the next sections for each energy variable, and possible improvements are discussed in section 5.

Electricity demand
The demand models performance can be evaluated during the three building steps (trend estimation, training and validation). Table 2 gives the error metrics of the model for France for each of the three periods displayed in Figure 3. The most significant metrics are the validation errors, calculated on an independent period when the models parameters have been determined after trend estimation and training. For the case of France, the MAPE is 1.17%, which is higher but very close to what is generally obtained with operational demand forecasting models. Table S3 provides the exact periods of training and validation for all the countries, as well as the corresponding metrics (RMSE, MAPE and correlation coefficient with ENTSO-E PS data).   The scatter plots of reconstructed versus actual load for the period (2015-2018) are presented in Figure   9 for four countries and in the Supplementary Material Figure S4 for  to get better quality data or fix issues in ENTSO-E data, but was out of scope of the project.

The Supplementary Material
Overall, the good performance of the GAM models lies in practice in three main aspects: i) the quality of the training data from the ENTSO-E database; ii) the degree of dependence of demand on the climate parameters and iii) the expert knowledge put in the model parameters definition.
Point ii) refers to the fact that some countries have more direct dependence of their demand on climate parameters, the most important one being temperature. France is the perfect example of this, as winter peak demand dependance is around 2,400 MW per degree Celsius. This is the highest dependance in Europe, where the total sensitivity of demand on temperature is around 4,000 MW per degree Celsius.
For those countries where climate data play a significant role, point iii) above relates to the degree of sophistication that was put into each model. Some optimization was made by using different combinations of climate variables (for instance, taking into account solar radiation in winter only), rsulting in an increase of the performance metrics. Close interaction with national experts could be mobilized to better tune each individual model and further improve their quality. Again, this is out of scope for the present work, but might be a promising action for further work. It has to be reminded that the simulated electricity demand has been detrended in the first step of the model set up. Therefore, the final data from 1979 to present reflects the variability due to the climate variables (and calendar data) only, and not the evolution of other exogeneous factors such as population changes and economic activity growth. The average level of demand, for instance expressed as an annual mean, is then similar to that of the end the training period. Should a particular user be interested to reconstruct actual demand (including population and GDP effects for instance), they should rescale the present data using for instance actual mean annual values of demand, which can be obtained from Eurostat or the World Bank.

Hydropower
As said in Section 3.2, modelling of hydro power has been particularly challenging, mostly due to the lack of data and information for calibration and validation purposes. A more extensive discussion of the results can be found in [30]. Table 3 and Table 4 show the correlation coefficient and the absolute errors (including a normalized version) for all the countries with hydro power capacity, for HRE and HRO respectively. Normalised MAE is defined as the MAE divided by the average generation on the entire period.  In general, the skills of hydro power run-of-river models are better than that of reservoir hydro power. This is because the majority of hydro power run-of-river plants are unregulated, depending then mainly on the meteorological conditions. Reservoir power plants are in general dispatchable then their generation is planned by the electric utilities considering also power systems conditions (prices, balancing needs, etc.). These effects are not taken into account in our models, which explains the lower performance for reservoirs based hydro power.
The models also perform better in countries with large energy production, such as reservoir type in France and Norway, and run-of-river in Italy. One possible explanation is that in countries with small installed capacity, energy production would have smaller signal-to-noise ratio, i.e. the variability has a similar order of magnitude to the relationship with meteorological conditions. Another reason is that we use country aggregated climate variables (temperature and precipitation); small installed capacity most often comes from a limited number of plants, which are located in specific locations of the country, for which the country average climate variables might not be very representative. The positive association between correlation and installed capacity of countries can be seen in Figure 6 and also in Table 3 and 4.  These results might also be explained by some limitation of the random forest model and machine learning models in general. In particular, the record length of the observed data used to train the models is very limited (only five years), whereas hydropower generation is characterized by a significant interannual variability. There is no doubt the models' accuracy will improve when additional data is available and models are retrained. Lastly, the relationship between temperature, precipitation and runof-river is rather obsious and direct, while the generation from reservoirs is also strongly influenced by the way reservoirs are managed, depending on the optimization made by network operators and dams' managers. These aspects have not been taken into account here, and are most likely the main explanation of the accuracy difference between run-of-river and reservoirs models.

Wind power
As mentioned in section 4.1, an issue to be dealt with when comparing model output with ENTSO-E data is the lack of information on the installed capacity. The resulting uncertainty hinders a detailed quantitative analysis of the model error. Thus a qualitative analysis is conducted by representing in Figure 11 the model output (capacity factor) for onshore wind power as a function of the ENTSO-E power data where the colours of the scatter points represent the year. The actual installed capacity increasing with time, we observe the slope of the scatter points in Figure 11   We can observe for some countries, for instance Denmark (but also Belgium and the Netherlands, see Supplementary figure S6), a saturation of the capacity factor when the wind speed exceeds the nominal wind speed of the chosen reference wind turbines (the nominal wind speed is the value over which the power production is maximum, hence corresponding to a capacity factor of 1.0). This effect is a direct consequence of the chosen modelling approach (use of a single wind turbine). It is particularly marked in small countries where the chosen power curve differs from that of the prevailing wind turbines.
Denmark and the Netherlands are two pioneer countries in wind energy. As a result, a large proportion of their park contains old turbines, with a lower hub height and a smaller nominal power than the ones chosen here and presented in section 3.4. It is therefore obvious that the representativity of a modern wind turbine used in our model is limited for those countries. Future evolution of the service will include the possibility to choose different types of wind turbines to better represent the diversity of actual insytalled machines. The spatial extension of a country is also playing a role in the presence of such effects in the country average: indeed the larger the averaging area, the more such effects will be smoothed out during the agregation. In countries like The Netherlands and Denmark, the smoothing effect is limited and the saturation is observable. A further reason may lie in the fact that ERA5 has some biases in wind speed as shown for instance in [39], with overall an overestimation in Northern Europe, and an underestimation in the South. The positive wind speed bias in the North artificially leads to an overestimation of the capacity factor for the Netherlands and Denmark. Yet, possible simulation errors due to bias in ERA5 are difficult to diagnose due to the uncertainty on the installed capacity and will be investigated in detail in the future.
Correlation coefficients for Switzerland and Austria (0.56 and 0.80 respectively) are low in comparison to the performances obtained for other countries (see correlation coefficients in Supplementary Material Table S2). As the actual location of wind farms has not been taken into account in this analysis, we assumed an homogeneous distribution of the capacity over the territory. It was shown in [38] that this assumption is reasonable for France and Germany; however, it is less relevant for countries located in the Alpine region. For these, there is little or even no wind turbines installed in mountainous areas, the assumption of uniform geographical distribution leads to large estimation errors. This issue is planned to be fixed in future versions where a more sophisticated aggregation approach will be implemented, using more realistic wind farms location assumptions.
With exception of the cases detailed in this section, most values of the correlation coefficients are greater than 0.8 for all countries, which is a very encouraging result that supports the plausibility of our onshore wind power model. The model might however be improved in three ways: first, by considering actual wind farms characteristics (location, technology, hub height…), second by using a more realistic approach for the spatial distribution of the installed capacity and third by applying a bias adjustment to the ERA5 wind speed data.

Solar PV
The same validation procedure used for onshore wind has been applied to the outputs of the solar PV model. The scatter plots representing the output of our model against power data provided by ENTSO-E are displayed again for four selected countries in Figure 12. A similar figure for all countries is available as Supplementary material Figure S8. As mentioned previously, the colour of the scatter poins represents the year. This representation is used to avoid the uncertainty on the installed capacity in the validation. As is the case for onshore wind, the approach cconsidering an homogeneous share of the installed capacity over the whole are of each country yields estimation errors in the Alpine region (Switzerland, Austria) but also in the United Kingdom. Larger estimation errors are also found for Spain. As identified in previous works, errors in Spain are explained by the fact that the ENTSO-E data encompass both PV and CSP (Concentrating Solar Power) while our model only considers PV systems [40]. Figure 6 showed in general good performances. Below-average performance is visible for the Netherlands, whose origin has not been identified so far and is still under investigation. Apart from the above mentioned cases, the values of the correlation coefficient are above 0.9 for all countries, which indicates that the output of the solar PV model is plausible.
The solar PV model might then be improved in two ways: first, as for wind onshore, by improving the assumption on the spatial distribution of the installed capacity and secondly by improving the countryspecific model weights to increase the correlation with the ENTSO-E data.

Discussion and conclusion
The objective of C3S Energy service is to produce a dataset of coherent weather-dependant energy variables for all European countries for different temporal scales: covering the last four decades, for the next 6-month using seasonal forecasts and until the end of the century using climate projections. In this paper, we describe the energy conversion models developed to generate this dataset. These are trained and validated against power data from the ENTSO-E databases. This choice was made to cover as many countries as possible, also considering the need to use publicly available data to comply with C3S open access policy.
The validation is conducted using reanalysis data as input to the models. Although not presented here, the same models have also been applied to seasonal forecasts from three of the C3S models (ECMWF, Météo-France and the UK Met Office) as well as to ten climate projections models from the EURO-CORDEX experiment [41,42] for IPCC scenarios RCP4.5 and RCP8.5.
The extended evaluation that was performed shows that the energy conversion models produce plausible demand and supply data, and in particular they reproduce fairly well the effects of climate variability on electricity demand and generation, noting that validation is hindered by uncertainty in the reference data.
Several causes have been identified to explain the model limitations, and improvement options have been listed, which might be implemented in the near future.
Firstly, energy observations are a key issue in developing and validating such models. The ENTSO-E TP database provides uniform access to data from all European countries only since 2015. The record length is relatively short especially with respect to the amount of data needed to calibrate statistical models when the interannual variability is large, as is the case for hydropower. In addition, issues were observed in the data and some inconsistencies have also been found. In order to increase the data quality and extend the depth of the archive, it is of utmost importance that significant effort is put is maintaining and improving the collection of better quality data on energy demand and generation, and facilitating the access to these data.
Secondly, the ERA5 climate data has been used here without any bias adjustment. Results from other research groups have shown that there are some biases, especially in wind speed. Further developments should implement bias adjustment on wind speed fields, which should significantly improve the wind power estimates.
Thirdly, energy conversion models could be improved. The wind power model is the most basic of all the models used here and could be enhanced for instance by using a more generic and adaptable power curve like in [43]. The wind and solar PV power estimates could also be improved by taking into account the actual location of the installed capacity. However, in the context of this service contract, it has to be remembered that the same models are applied for projections until the end of the century, when the location of wind and solar capacity cannot be anticipated. However, interested users could consider integrating their own scenarios providing the evolution of the installed capacity of each country in the future. The hydropower model could also be improved, and more collaboration could be sought with hydrological model developers into the future, to possibly find intermediate complexity models, between full hydrological models and the more simple models used here. The demand models could also be adapted to the hourly time resolution, in order to fit the needs of adequacy studies. It could also benefit from direct interactions with national TSOs, in order to refine the models equations based on their expertise of the demand behovior in their country.

Data Availability
All processed C3S-E datasets are passed to a publicly accessible drive. Then, as part of a monthly automated procedure, integrated into the Climate Data Store (CDS). The dataset is freely available to download from the CDS 8 to anyone who has a registered account (DOI: 10.24381/cds.4bd77450). In addition to downloading the data from the CDS, users are able to access and plot data via the online visualisation tool 9 , made available as part of the CDS toolbox.  The proposed approach is motivated by the physics but its final form is the result of different tries which are not described here. The set of coefficient ; #$ has been evaluated for each grid cell independently and saved in a netdcf parameter file. To illustrate the dependency of the solar capacity factor on the selected features, two examples corresponding to a location in the North and in the South of the C3S area are given in Figure S1 and Figure S2 (the location is represented in the left plots of the two figures). In the middle plot of the two figures, the dependency of the capacity factor as a function of the GHI and TOA irradiation -the two most important predictors -is represented. It can be observed in these two examples that the cloud of scatter points exhibits a clear dependency on these two parameters and that this dependency varies spatially. Scatter plots of the regression as a function of the reference data are given for the two examples in the right plots of Figure S1 and Figure S2 : Illustration of the linear model for a grid point located in the south of the considered area (35°N,5°E). It can be observed that the regression yields very small residuals yet slightly larger in Northern regions.
A potential drawback of the proposed approach is that it can yield abnormal result if the input parameters are out of the domain covered by the set of training data. To increase its robustness, outputs of the model are constrained by the two following bounds, which are represented by a dotted line in the middle plots of Figure S1 and Figure   S2  The proposed approach can be directly implemented at daily resolution but needs some adaptation to handle the 3-hourly time resolution of the climate projections. In this latter case we propose to evaluate the solar capacity factors using daily data, as described above, and subsequently downscale the data at a three-hourly resolution by assuming that the time evolution of the solar capacity factor is proportional to that of the GHI.