Articles | Volume 15, issue 6
https://doi.org/10.5194/essd-15-2635-2023
© Author(s) 2023. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
https://doi.org/10.5194/essd-15-2635-2023
© Author(s) 2023. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Data description paper
| 
28 Jun 2023
Data description paper |
| 28 Jun 2023
| 28 Jun 2023
The EUPPBench postprocessing benchmark dataset v1.0
Jonathan Demaeyer
CORRESPONDING AUTHOR
Meteorological and Climatological Information Service, Royal Meteorological Institute of Belgium, Brussels, Belgium
European Meteorological Network (EUMETNET), Brussels, Belgium
Jonas Bhend
Federal Office of Meteorology and Climatology MeteoSwiss, Zurich, Switzerland
Sebastian Lerch
Institute of Economics, Karlsruhe Institute of Technology, Karlsruhe, Germany
Cristina Primo
Deutscher Wetterdienst, Offenbach, Germany
Bert Van Schaeybroeck
Meteorological and Climatological Information Service, Royal Meteorological Institute of Belgium, Brussels, Belgium
Aitor Atencia
GeoSphere Austria, Vienna, Austria
Zied Ben Bouallègue
European Centre for Medium-Range Weather Forecasts, Reading, United Kingdom
Jieyu Chen
Institute of Economics, Karlsruhe Institute of Technology, Karlsruhe, Germany
Markus Dabernig
GeoSphere Austria, Vienna, Austria
Gavin Evans
Met Office, Exeter, United Kingdom
Jana Faganeli Pucer
Faculty of Computer and Information Science, University of Ljubljana, Slovenia
Ben Hooper
Met Office, Exeter, United Kingdom
Nina Horat
Institute of Economics, Karlsruhe Institute of Technology, Karlsruhe, Germany
David Jobst
Institute of Mathematics and Applied Informatics, University of Hildesheim, Hildesheim, Germany
Janko Merše
Slovenian Environment Agency, Ljubljana, Slovenia
Peter Mlakar
Faculty of Computer and Information Science, University of Ljubljana, Slovenia
Slovenian Environment Agency, Ljubljana, Slovenia
Annette Möller
Faculty of Business Administration and Economics, Bielefeld University, Bielefeld, Germany
Olivier Mestre
CNRM, Université de Toulouse, Météo-France, CNRS, Toulouse, France
Météo-France, Toulouse, France
Maxime Taillardat
CNRM, Université de Toulouse, Météo-France, CNRS, Toulouse, France
Météo-France, Toulouse, France
Stéphane Vannitsem
Meteorological and Climatological Information Service, Royal Meteorological Institute of Belgium, Brussels, Belgium
European Meteorological Network (EUMETNET), Brussels, Belgium
Related authors
Elke Debrie, Jonathan Demaeyer, and Stéphane Vannitsem
Earth Syst. Sci. Data Discuss., https://doi.org/10.5194/essd-2025-149, https://doi.org/10.5194/essd-2025-149, 2025
Revised manuscript under review for ESSD
Short summary
Short summary
In this project, we developed a gridded hourly precipitation dataset for Belgium, covering over 70 years (1940–2016). The data has a spatial resolution of one kilometer, which means it provides highly localized precipitation information. To estimate precipitation for a specific day in the past, we searched for days in the recent radar data period with similar weather patterns, known as the analog method. The median of the produced dataset is available for public use and can be found on Zenodo.
Anupama K. Xavier, Jonathan Demaeyer, and Stéphane Vannitsem
Earth Syst. Dynam., 15, 893–912, https://doi.org/10.5194/esd-15-893-2024, https://doi.org/10.5194/esd-15-893-2024, 2024
Short summary
Short summary
This research focuses on understanding different atmospheric patterns like blocking, zonal, and transition regimes and analyzing their predictability. We used an idealized land–atmosphere coupled model to simulate Earth's atmosphere. Then we identified these blocking, zonal, and transition regimes using Gaussian mixture clustering and studied their predictability using Lyapunov exponents.
Martin Bonte and Stéphane Vannitsem
Nonlin. Processes Geophys., 32, 139–165, https://doi.org/10.5194/npg-32-139-2025, https://doi.org/10.5194/npg-32-139-2025, 2025
Short summary
Short summary
In recent years, there have been more and more floods due to intense precipitation, such as the July 2021 event in Belgium. Predicting precipitation is a difficult task, even just for the next few hours. This study focuses on a tool that assesses whether a given situation is stable or not (i.e., whether it is likely to stay as it is or could evolve in an unpredictable manner).
Stéphane Vannitsem, X. San Liang, and Carlos A. Pires
Earth Syst. Dynam., 16, 703–719, https://doi.org/10.5194/esd-16-703-2025, https://doi.org/10.5194/esd-16-703-2025, 2025
Short summary
Short summary
Large-scale modes of variability are present in the climate system. These modes are known to have influences on each other but are usually viewed as linear influences. The nonlinear connections among a set of key climate indices are explored here using tools from information theory, which allow us to characterize the causality between indices. It was found that quadratic nonlinear dependencies between climate indices are present at low frequencies, reflecting the complex nature of their dynamics.
Elke Debrie, Jonathan Demaeyer, and Stéphane Vannitsem
Earth Syst. Sci. Data Discuss., https://doi.org/10.5194/essd-2025-149, https://doi.org/10.5194/essd-2025-149, 2025
Revised manuscript under review for ESSD
Short summary
Short summary
In this project, we developed a gridded hourly precipitation dataset for Belgium, covering over 70 years (1940–2016). The data has a spatial resolution of one kilometer, which means it provides highly localized precipitation information. To estimate precipitation for a specific day in the past, we searched for days in the recent radar data period with similar weather patterns, known as the analog method. The median of the produced dataset is available for public use and can be found on Zenodo.
Romain Pic, Clément Dombry, Philippe Naveau, and Maxime Taillardat
Adv. Stat. Clim. Meteorol. Oceanogr., 11, 23–58, https://doi.org/10.5194/ascmo-11-23-2025, https://doi.org/10.5194/ascmo-11-23-2025, 2025
Short summary
Short summary
Correctly forecasting weather is crucial for decision-making in various fields. Standard multivariate verification tools have limitations, and a single tool cannot fully characterize predictive performance. We formalize a framework based on aggregation and transformation to build interpretable verification tools. These tools target specific features of forecasts, improving predictive performance characterization and bridging the gap between theoretical and physics-based tools.
Yanwei Zhu, Aitor Atencia, Markus Dabernig, and Yong Wang
Geosci. Model Dev., 18, 1545–1559, https://doi.org/10.5194/gmd-18-1545-2025, https://doi.org/10.5194/gmd-18-1545-2025, 2025
Short summary
Short summary
Most works have delved into convective weather nowcasting, and only a few works have discussed the nowcasting uncertainty for variables at the surface level. Hence, we proposed a method to estimate uncertainty. Generating appropriate noises associated with the characteristic of the error in analysis can simulate the uncertainty of nowcasting. This method can contribute to the estimation of near–surface analysis uncertainty in both nowcasting applications and ensemble nowcasting development.
Anouk Dierickx, Wout Dewettinck, Bert Van Schaeybroeck, Lesley De Cruz, Steven Caluwaerts, Piet Termonia, and Hans Van de Vyver
Earth Syst. Sci. Data Discuss., https://doi.org/10.5194/essd-2025-30, https://doi.org/10.5194/essd-2025-30, 2025
Preprint under review for ESSD
Short summary
Short summary
This study introduces the EURO-SUPREME dataset consisting of extreme precipitation events selected from a large ensemble of climate models over Europe. The dataset contains information on extreme precipitation events with a precipitation duration of 1 hour to 72 hours that can lead to flooding, high mortality rates and infrastructure damage. We highlight the usefulness of the dataset as a benchmark for improving high-resolution climate models for risk assessment of future extreme floods.
Selina M. Kiefer, Patrick Ludwig, Sebastian Lerch, Peter Knippertz, and Joaquim G. Pinto
EGUsphere, https://doi.org/10.5194/egusphere-2024-2955, https://doi.org/10.5194/egusphere-2024-2955, 2024
Preprint withdrawn
Short summary
Short summary
Weather forecasts 14 days in advance generally have a low skill but not always. We identify reasons thereof depending on the atmospheric flow, shown by Weather Regimes (WRs). If the WRs during the forecasts follow climatological patterns, forecast skill is increased. The forecast of a cold-wave day is better when the European Blocking WR (high pressure around the British Isles) is present a few days before a cold-wave day. These results can be used to assess the reliability of predictions.
Anupama K. Xavier, Jonathan Demaeyer, and Stéphane Vannitsem
Earth Syst. Dynam., 15, 893–912, https://doi.org/10.5194/esd-15-893-2024, https://doi.org/10.5194/esd-15-893-2024, 2024
Short summary
Short summary
This research focuses on understanding different atmospheric patterns like blocking, zonal, and transition regimes and analyzing their predictability. We used an idealized land–atmosphere coupled model to simulate Earth's atmosphere. Then we identified these blocking, zonal, and transition regimes using Gaussian mixture clustering and studied their predictability using Lyapunov exponents.
Loris Foresti, Bernat Puigdomènech Treserras, Daniele Nerini, Aitor Atencia, Marco Gabella, Ioannis V. Sideris, Urs Germann, and Isztar Zawadzki
Nonlin. Processes Geophys., 31, 259–286, https://doi.org/10.5194/npg-31-259-2024, https://doi.org/10.5194/npg-31-259-2024, 2024
Short summary
Short summary
We compared two ways of defining the phase space of low-dimensional attractors describing the evolution of radar precipitation fields. The first defines the phase space by the domain-scale statistics of precipitation fields, such as their mean, spatial and temporal correlations. The second uses principal component analysis to account for the spatial distribution of precipitation. To represent different climates, radar archives over the United States and the Swiss Alpine region were used.
Peter Mlakar, Antonio Ricchi, Sandro Carniel, Davide Bonaldo, and Matjaž Ličer
Geosci. Model Dev., 17, 4705–4725, https://doi.org/10.5194/gmd-17-4705-2024, https://doi.org/10.5194/gmd-17-4705-2024, 2024
Short summary
Short summary
We propose a new point-prediction model, the DEep Learning WAVe Emulating model (DELWAVE), which successfully emulates the Simulating WAves Nearshore model (SWAN) over synoptic to climate timescales. Compared to control climatology over all wind directions, the mismatch between DELWAVE and SWAN is generally small compared to the difference between scenario and control conditions, suggesting that the noise introduced by surrogate modelling is substantially weaker than the climate change signal.
David Docquier, Giorgia Di Capua, Reik V. Donner, Carlos A. L. Pires, Amélie Simon, and Stéphane Vannitsem
Nonlin. Processes Geophys., 31, 115–136, https://doi.org/10.5194/npg-31-115-2024, https://doi.org/10.5194/npg-31-115-2024, 2024
Short summary
Short summary
Identifying causes of specific processes is crucial in order to better understand our climate system. Traditionally, correlation analyses have been used to identify cause–effect relationships in climate studies. However, correlation does not imply causation, which justifies the need to use causal methods. We compare two independent causal methods and show that these are superior to classical correlation analyses. We also find some interesting differences between the two methods.
Michel Journée, Edouard Goudenhoofdt, Stéphane Vannitsem, and Laurent Delobbe
Hydrol. Earth Syst. Sci., 27, 3169–3189, https://doi.org/10.5194/hess-27-3169-2023, https://doi.org/10.5194/hess-27-3169-2023, 2023
Short summary
Short summary
The exceptional flood of July 2021 in central Europe impacted Belgium severely. This study aims to characterize rainfall amounts in Belgium from 13 to 16 July 2021 based on observational data (i.e., rain gauge data and a radar-based rainfall product). The spatial and temporal distributions of rainfall during the event aredescribed. In order to document such a record-breaking event as much as possible, the rainfall data are shared with the scientific community on Zenodo for further studies.
David Docquier, Stéphane Vannitsem, and Alessio Bellucci
Earth Syst. Dynam., 14, 577–591, https://doi.org/10.5194/esd-14-577-2023, https://doi.org/10.5194/esd-14-577-2023, 2023
Short summary
Short summary
The climate system is strongly regulated by interactions between the ocean and atmosphere. However, many uncertainties remain in the understanding of these interactions. Our analysis uses a relatively novel approach to quantify causal links between the ocean surface and lower atmosphere based on satellite observations. We find that both the ocean and atmosphere influence each other but with varying intensity depending on the region, demonstrating the power of causal methods.
Stéphane Vannitsem
Nonlin. Processes Geophys., 30, 1–12, https://doi.org/10.5194/npg-30-1-2023, https://doi.org/10.5194/npg-30-1-2023, 2023
Short summary
Short summary
The impact of climate change on weather pattern dynamics over the North Atlantic is explored through the lens of information theory. These tools allow the predictability of the succession of weather patterns and the irreversible nature of the dynamics to be clarified. It is shown that the predictability is increasing in the observations, while the opposite trend is found in model projections. The irreversibility displays an overall increase in time in both the observations and the model runs.
David Docquier, Stéphane Vannitsem, Alessio Bellucci, and Claude Frankignoul
EGUsphere, https://doi.org/10.5194/egusphere-2022-1340, https://doi.org/10.5194/egusphere-2022-1340, 2022
Preprint withdrawn
Short summary
Short summary
Understanding whether variations in ocean heat content are driven by air-sea heat fluxes or by ocean dynamics is of crucial importance to enhance climate projections. We use a relatively novel causal method to quantify interactions between ocean heat budget terms based on climate models. We find that low-resolution models overestimate the influence of ocean dynamics in the upper ocean, and that changes in ocean heat content are dominated by air-sea fluxes at high resolution.
Riccardo Silini, Sebastian Lerch, Nikolaos Mastrantonas, Holger Kantz, Marcelo Barreiro, and Cristina Masoller
Earth Syst. Dynam., 13, 1157–1165, https://doi.org/10.5194/esd-13-1157-2022, https://doi.org/10.5194/esd-13-1157-2022, 2022
Short summary
Short summary
The Madden–Julian Oscillation (MJO) has important socioeconomic impacts due to its influence on both tropical and extratropical weather extremes. In this study, we use machine learning (ML) to correct the predictions of the weather model holding the best performance, developed by the European Centre for Medium-Range Weather Forecasts (ECMWF). We show that the ML post-processing leads to an improved prediction of the MJO geographical location and intensity.
Núria Pérez-Zanón, Louis-Philippe Caron, Silvia Terzago, Bert Van Schaeybroeck, Llorenç Lledó, Nicolau Manubens, Emmanuel Roulin, M. Carmen Alvarez-Castro, Lauriane Batté, Pierre-Antoine Bretonnière, Susana Corti, Carlos Delgado-Torres, Marta Domínguez, Federico Fabiano, Ignazio Giuntoli, Jost von Hardenberg, Eroteida Sánchez-García, Verónica Torralba, and Deborah Verfaillie
Geosci. Model Dev., 15, 6115–6142, https://doi.org/10.5194/gmd-15-6115-2022, https://doi.org/10.5194/gmd-15-6115-2022, 2022
Short summary
Short summary
CSTools (short for Climate Service Tools) is an R package that contains process-based methods for climate forecast calibration, bias correction, statistical and stochastic downscaling, optimal forecast combination, and multivariate verification, as well as basic and advanced tools to obtain tailored products. In addition to describing the structure and methods in the package, we also present three use cases to illustrate the seasonal climate forecast post-processing for specific purposes.
Nicolas Ghilain, Stéphane Vannitsem, Quentin Dalaiden, Hugues Goosse, Lesley De Cruz, and Wenguang Wei
Earth Syst. Sci. Data, 14, 1901–1916, https://doi.org/10.5194/essd-14-1901-2022, https://doi.org/10.5194/essd-14-1901-2022, 2022
Short summary
Short summary
Modeling the climate at high resolution is crucial to represent the snowfall accumulation over the complex orography of the Antarctic coast. While ice cores provide a view constrained spatially but over centuries, climate models can give insight into its spatial distribution, either at high resolution over a short period or vice versa. We downscaled snowfall accumulation from climate model historical simulations (1850–present day) over Dronning Maud Land at 5.5 km using a statistical method.
Gerard van der Schrier, Richard P. Allan, Albert Ossó, Pedro M. Sousa, Hans Van de Vyver, Bert Van Schaeybroeck, Roberto Coscarelli, Angela A. Pasqua, Olga Petrucci, Mary Curley, Mirosław Mietus, Janusz Filipiak, Petr Štěpánek, Pavel Zahradníček, Rudolf Brázdil, Ladislava Řezníčková, Else J. M. van den Besselaar, Ricardo Trigo, and Enric Aguilar
Clim. Past, 17, 2201–2221, https://doi.org/10.5194/cp-17-2201-2021, https://doi.org/10.5194/cp-17-2201-2021, 2021
Short summary
Short summary
The 1921 drought was the most severe drought to hit Europe since the start of the 20th century. Here the climatological description of the drought is coupled to an overview of its impacts, sourced from newspapers, and an analysis of its drivers. The area from Ireland to the Ukraine was affected but hardest hit was the triangle between Brussels, Paris and Lyon. The drought impacts lingered on until well into autumn and winter, affecting water supply and agriculture and livestock farming.
Guillaume Evin, Matthieu Lafaysse, Maxime Taillardat, and Michaël Zamo
Nonlin. Processes Geophys., 28, 467–480, https://doi.org/10.5194/npg-28-467-2021, https://doi.org/10.5194/npg-28-467-2021, 2021
Short summary
Short summary
Forecasting the height of new snow is essential for avalanche hazard surveys, road and ski resort management, tourism attractiveness, etc. Météo-France operates a probabilistic forecasting system using a numerical weather prediction system and a snowpack model. It provides better forecasts than direct diagnostics but exhibits significant biases. Post-processing methods can be applied to provide automatic forecasting products from this system.
Tommaso Alberti, Reik V. Donner, and Stéphane Vannitsem
Earth Syst. Dynam., 12, 837–855, https://doi.org/10.5194/esd-12-837-2021, https://doi.org/10.5194/esd-12-837-2021, 2021
Short summary
Short summary
We provide a novel approach to diagnose the strength of the ocean–atmosphere coupling by using both a reduced order model and reanalysis data. Our findings suggest the ocean–atmosphere dynamics presents a rich variety of features, moving from a chaotic to a coherent coupled dynamics, mainly attributed to the atmosphere and only marginally to the ocean. Our observations suggest further investigations in characterizing the occurrence and spatial dependency of the ocean–atmosphere coupling.
Sara Top, Lola Kotova, Lesley De Cruz, Svetlana Aniskevich, Leonid Bobylev, Rozemien De Troch, Natalia Gnatiuk, Anne Gobin, Rafiq Hamdi, Arne Kriegsmann, Armelle Reca Remedio, Abdulla Sakalli, Hans Van De Vyver, Bert Van Schaeybroeck, Viesturs Zandersons, Philippe De Maeyer, Piet Termonia, and Steven Caluwaerts
Geosci. Model Dev., 14, 1267–1293, https://doi.org/10.5194/gmd-14-1267-2021, https://doi.org/10.5194/gmd-14-1267-2021, 2021
Short summary
Short summary
Detailed climate data are needed to assess the impact of climate change on human and natural systems. The performance of two high-resolution regional climate models, ALARO-0 and REMO2015, was investigated over central Asia, a vulnerable region where detailed climate information is scarce. In certain subregions the produced climate data are suitable for impact studies, but bias adjustment is required for subregions where significant biases have been identified.
Stephan Hemri, Sebastian Lerch, Maxime Taillardat, Stéphane Vannitsem, and Daniel S. Wilks
Nonlin. Processes Geophys., 27, 519–521, https://doi.org/10.5194/npg-27-519-2020, https://doi.org/10.5194/npg-27-519-2020, 2020
Cited articles
Ashkboos, S., Huang, L., Dryden, N., Ben-Nun, T., Dueben, P., Gianinazzi, L.,
Kummer, L., and Hoefler, T.: Ens-10: A dataset for post-processing ensemble
weather forecast, arXiv [preprint], https://doi.org/10.48550/arXiv.2206.14786, 29 June 2022. a, b
Ben Bouallègue, Z.: Accounting for representativeness in the verification
of ensemble forecasts, ECMWF Technical Memoranda, 865, https://doi.org/10.21957/5z6esc7wr, 2020. a
Ben Bouallègue, Z.: EUPP-benchmark/ESSD-ASRE: version 1.0 release, Zenodo [code],
https://doi.org/10.5281/zenodo.7477735, 2023. a
Benjamini, Y. and Hochberg, Y.: Controlling the False Discovery Rate: A
Practical and Powerful Approach to Multiple Testing, J. Roy.
Stat. Soc. B Met., 57, 289–300, https://doi.org/10.1111/j.2517-6161.1995.tb02031.x, 1995. a
Bhend, J., Dabernig, M., Demaeyer, J., Mestre, O., and Taillardat, M.:
EUPPBench postprocessing benchmark dataset – station data, Zenodo [data set],
https://doi.org/10.5281/zenodo.7708362, 2023. a, b
Bremnes, J. B.: Ensemble postprocessing using quantile function regression
based on neural networks and Bernstein polynomials, Mon. Weather Rev.,
148, 403–414, 2020. a
Chapman, W. E., Monache, L. D., Alessandrini, S., Subramanian, A. C., Ralph,
F. M., Xie, S.-P., Lerch, S., and Hayatbini, N.: Probabilistic Predictions
from Deterministic Atmospheric River Forecasts with Deep Learning, Mon.
Weather Rev., 150, 215–234, https://doi.org/10.1175/MWR-D-21-0106.1, 2022. a
Chen, J., Janke, T., Steinke, F., and Lerch, S.: Generative machine learning
methods for multivariate ensemble post-processing, arXiv [preprint],
https://doi.org/10.48550/arXiv.2211.01345, 26 September 2022. a
Chen, J., Dabernig, M., Demaeyer, J., Evans, G., Faganeli Pucer, J., Hooper,
B., Horat, N., Jobst, D., Lerch, S., Mlakar, P., Möller, A., Merše, J., and
Bouallègue, Z. B.: ESSD benchmark output data, Zenodo [data set], https://doi.org/10.5281/zenodo.7798350,
2023a. a
Chen, J., Horat, N., and Lerch, S.: EUPP-benchmark/ESSD-DRN: version 1.0
release, Zenodo [code], https://doi.org/10.5281/zenodo.7477698, 2023b. a
Copernicus Land Monitoring Service, E. U.: EU-DEM, European
Environment Agency, CLC
https://land.copernicus.eu/imagery-in-situ/eu-dem, last access: 1 August 2022. a
Dabernig, M.: EUPP-benchmark/ESSD-EMOS: version 1.0 release, Zenodo [code],
https://doi.org/10.5281/zenodo.7477749, 2023. a
Dabernig, M., Mayr, G. J., Messner, J. W., and Zeileis, A.: Spatial Ensemble
Post-Processing with Standardized Anomalies, Q. J. Roy.
Meteor. Soc., 143, 909–916, https://doi.org/10.1002/qj.2975,
2017. a
Demaeyer, J.: Climdyn/pythie: Version 0.1.0 alpha release, Zenodo [code],
https://doi.org/10.5281/zenodo.7233538, 2022a. a
Demaeyer, J.: EUPPBench postprocessing benchmark dataset – gridded data – Part
I, Zenodo [data set], https://doi.org/10.5281/zenodo.7429236, 2022b. a, b
Demaeyer, J.: EUPP-benchmark/ESSD-mbm: version 1.0 release, Zenodo [code],
https://doi.org/10.5281/zenodo.7476673, 2023. a
Demaeyer, J. and Vannitsem, S.: Correcting for model changes in statistical postprocessing – an approach based on response theory, Nonlin. Processes Geophys., 27, 307–327, https://doi.org/10.5194/npg-27-307-2020, 2020. a
Dueben, P. D., Schultz, M. G., Chantry, M., Gagne, D. J., Hall, D. M., and
McGovern, A.: Challenges and Benchmark Datasets for Machine Learning in the
Atmospheric Sciences: Definition, Status, and Outlook, Artificial
Intelligence for the Earth Systems, 1, e210002,
https://doi.org/10.1175/AIES-D-21-0002.1, 2022. a
Eaton, B., Gregory, J., Drach, B., Taylor, K., Hankin, S., Caron, J., Signell,
R., Bentley, P., Rappa, G., Höck, H., Pamment, A., Juckes, M., Raspaud, M., Horne, R.,
Whiteaker, T., Blodgett, D., Zender, C., and Lee, D.: NetCDF Climate and Forecast
(CF) metadata conventions,
http://cfconventions.org/Data/cf-conventions/cf-conventions-1.8/cf-conventions.pdf (last access: 2 June 2023),
2003. a
EUMETNET: EUPPBench datasets documentation, EUMETNET [data set], https://eupp-benchmark.github.io/EUPPBench-doc/files/EUPPBench_datasets.html, last access: 2023. a
Evans, G. and Hooper, B.: EUPP-benchmark/ESSD-reliability-calibration: version
1.0 release, Zenodo [code], https://doi.org/10.5281/zenodo.7476590, 2023. a
Flowerdew, J.: Calibrating ensemble reliability whilst preserving spatial
structure, Tellus A, 66, 22662,
https://doi.org/10.3402/tellusa.v66.22662, 2014. a, b
Garg, S., Rasp, S., and Thuerey, N.: WeatherBench Probability: A benchmark
dataset for probabilistic medium-range weather forecasting along with deep
learning baseline models, arXiv [preprint], https://doi.org/10.48550/arXiv.2205.00865, 2 May 2022. a
Glahn, H. R. and Lowry, D. A.: The use of model output statistics (MOS) in
objective weather forecasting, J. Appl. Meteorol.
Clim., 11, 1203–1211, 1972. a
Gregory, J.: The CF metadata standard, CLIVAR Exchanges, 8, 4, http://cfconventions.org/Data/cf-documents/overview/article.pdf (last access: June 2022), 2003. a
Groß, J. and Möller, A.: ensAR: Autoregressive postprocessing methods
for ensemble forecasts, R
package version 0.2.0, https://github.com/JuGross/ensAR (last access: 2 June 2023), 2019. a
Hafizi, H. and Sorman, A. A.: Assessment of 13 Gridded Precipitation Datasets
for Hydrological Modeling in a Mountainous Basin, Atmosphere, 13, 143,
https://doi.org/10.3390/atmos13010143, 2022. a
Hagedorn, R., Hamill, T. M., and Whitaker, J. S.: Probabilistic Forecast
Calibration Using ECMWF and GFS Ensemble Reforecasts. Part I: Two-Meter
Temperatures, Mon. Weather Rev., 136, 2608–2619,
https://doi.org/10.1175/2007MWR2410.1, 2008. a
Hamill, T. M., Hagedorn, R., and Whitaker, J. S.: Probabilistic Forecast
Calibration Using ECMWF and GFS Ensemble Reforecasts. Part II: Precipitation,
Mon. Weather Rev., 136, 2620–2632, https://doi.org/10.1175/2007MWR2411.1, 2008. a
Han, J., Miao, C., Gou, J., Zheng, H., Zhang, Q., and Guo, X.: A new daily gridded precipitation dataset based on gauge observations across mainland China, Earth Syst. Sci. Data Discuss. [preprint], https://doi.org/10.5194/essd-2022-373, in review, 2022. a
Haupt, S. E., Chapman, W., Adams, S. V., Kirkwood, C., Hosking, J. S.,
Robinson, N. H., Lerch, S., and Subramanian, A. C.: Towards implementing
artificial intelligence post-processing in weather and climate: proposed
actions from the Oxford 2019 workshop, Philos. T.
Roy. Soc. A, 379, 20200091, https://doi.org/10.1098/rsta.2020.0091, 2021. a, b
Hersbach, H.: Decomposition of the continuous ranked probability score for
ensemble prediction systems, Weather Forecast., 15, 559–570, 2000. a
Hersbach, H., Bell, B., Berrisford, P., Hirahara, S., Horányi, A.,
Muñoz-Sabater, J., Nicolas, J., Peubey, C., Radu, R., Schepers, D., Simmons,
A., Soci, C., Abdalla, S., Abellan, X., Balsamo, G., Bechtold, P., Biavati,
G., Bidlot, J., Bonavita, M., De Chiara, G., Dahlgren, P., Dee, D.,
Diamantakis, M., Dragani, R., Flemming, J., Forbes, R., Fuentes, M., Geer,
A., Haimberger, L., Healy, S., Hogan, R. J., Hólm, E., Janisková, M.,
Keeley, S., Laloyaux, P., Lopez, P., Lupu, C., Radnoti, G., de Rosnay, P.,
Rozum, I., Vamborg, F., Villaume, S., and Thépaut, J.-N.: The ERA5 global
reanalysis, Q. J. Roy. Meteor. Soc., 146,
1999–2049, https://doi.org/10.1002/qj.3803, 2020. a
Horton, P.: Analogue methods and ERA5: Benefits and pitfalls, Int.
J. Climatol., 42, 4078–4096,
https://doi.org/10.1002/joc.7484, 2022. a
Hoyer, S. and Joseph, H.: xarray: N-D labeled Arrays and Datasets in Python,
Journal of Open Research Software, 5, 10, https://doi.org/10.5334/jors.148, 2017. a
Jobst, D.: EUPP-benchmark/ESSD-DVQR: version 1.0 release, Zenodo [code],
https://doi.org/10.5281/zenodo.7477640, 2023. a
Kim, T., Ho, N., Kim, D., and Yun, S.-Y.: Benchmark Dataset for Precipitation
Forecasting by Post-Processing the Numerical Weather Prediction, arXiv
[preprint], https://doi.org/10.48550/arXiv.2206.15241, 30 June 2022. a
Klein, W. H. and Lewis, F.: Computer Forecasts of Maximum and Minimum Temperatures, J. Appl. Meteorol. Clim., 9, 350–359, https://doi.org/10.1175/1520-0450(1970)009<0350:CFOMAM>2.0.CO;2, 1970. a, b
Kraus, D. and Czado, C.: D-vine copula based quantile regression, Comput.
Stat. Data An., 110, 1–18, https://doi.org/10.1016/j.csda.2016.12.009,
2017. a
Lakatos, M., Lerch, S., Hemri, S., and Baran, S.: Comparison of multivariate
post-processing methods using global ECMWF ensemble forecasts, Q.
J. Roy. Meteor. Soc., 149, 856–877, https://doi.org/10.1002/qj.4436, 2023. a
Lalaurette, F.: Early detection of abnormal weather conditions using a
probabilistic extreme forecast index, Q. J. Roy.
Meteor. Soc., 129, 3037–3057, 2003. a
Lang, M. N., Lerch, S., Mayr, G. J., Simon, T., Stauffer, R., and Zeileis, A.: Remember the past: a comparison of time-adaptive training schemes for non-homogeneous regression, Nonlin. Processes Geophys., 27, 23–34, https://doi.org/10.5194/npg-27-23-2020, 2020. a
Lenkoski, A., Kolstad, E. W., and Thorarinsdottir, T. L.: A Benchmarking
Dataset for Seasonal Weather Forecasts, NR-notat,
https://nr.brage.unit.no/nr-xmlui/bitstream/handle/11250/2976154/manual.pdf (last access: 22 December 2022),
2022. a
Lerch, S., Baran, S., Möller, A., Groß, J., Schefzik, R., Hemri, S., and Graeter, M.: Simulation-based comparison of multivariate ensemble post-processing methods, Nonlin. Processes Geophys., 27, 349–371, https://doi.org/10.5194/npg-27-349-2020, 2020. a
Maciel, P., Quintino, T., Modigliani, U., Dando, P., Raoult, B., Deconinck, W.,
Rathgeber, F., and Simarro, C.: The new ECMWF interpolation package MIR,
36–39, https://doi.org/10.21957/h20rz8, 2017. a
Messner, J. W., Mayr, G. J., and Zeileis, A.: Heteroscedastic Censored and
Truncated Regression with crch, The R Journal, 8, 173–181,
https://journal.r-project.org/archive/2016-1/messner-mayr-zeileis.pdf (last access: 2 June 2023),
2016. a
Miles, A., Kirkham, J., Durant, M., Bourbeau, J., Onalan, T., Hamman, J.,
Patel, Z., shikharsg, Rocklin, M., raphael dussin, Schut, V., de Andrade,
E. S., Abernathey, R., Noyes, C., sbalmer, pyup.io bot, Tran, T., Saalfeld,
S., Swaney, J., Moore, J., Jevnik, J., Kelleher, J., Funke, J., Sakkis, G.,
Barnes, C., and Banihirwe, A.: zarr-developers/zarr-python: v2.4.0, Zenodo [code],
https://doi.org/10.5281/zenodo.3773450, 2020. a
Mlakar, P., Merše, J., and Pucer, J. F.: Ensemble weather forecast
post-processing with a flexible probabilistic neural network approach, arXiv [preprint],
https://doi.org/10.48550/arXiv.2303.17610, 29 March 2023a. a
Mlakar, P., Pucer, J. F., and Merše, J.: EUPP-benchmark/ESSD-ANET: version 1.0
release, Zenodo [code], https://doi.org/10.5281/zenodo.7479333,
2023b. a
Möller, A.: EUPP-benchmark/ESSD-AR-EMOS: version 1.0 release, Zenodo [code],
https://doi.org/10.5281/zenodo.7477633, 2023. a
Möller, A. and Groß, J.: Probabilistic temperature forecasting based on
an ensemble autoregressive modification, Q. J. Roy.
Meteor. Soc., 142, 1385–1394,
https://doi.org/10.1002/qj.2741, 2016. a
Möller, A. and Groß, J.: Probabilistic Temperature Forecasting with a
Heteroscedastic Autoregressive Ensemble Postprocessing model, Q.
J. Roy. Meteor. Soc., 146, 211–224,
https://doi.org/10.1002/qj.3667, 2020. a
Möller, A., Spazzini, L., Kraus, D., Nagler, T., and Czado, C.: Vine copula
based post-processing of ensemble forecasts for temperature, arXiv [preprint], https://doi.org/10.48550/arXiv.1811.02255, 6 November 2018. a
Murphy, A. H.: What Is a Good Forecast? An Essay on the Nature of Goodness in
Weather Forecasting, Weather Forecast., 8, 281–293,
https://doi.org/10.1175/1520-0434(1993)008<0281:WIAGFA>2.0.CO;2, 1993. a
Nagler, T.: vinereg: D-Vine Quantile Regression, r package version
0.7.2, CRAN [code],
https://CRAN.R-project.org/package=vinereg (last access: 17 November 2020), 2020. a
Perrone, E., Schicker, I., and Lang, M. N.: A case study of empirical copula
methods for the statistical correction of forecasts of the ALADIN-LAEF
system, Meteorol. Z., 29, 277–288,
https://doi.org/10.1127/metz/2020/1034, 2020. a
Primo-Ramos, C., Bhend, J., Atencia, A., Van schaeybroeck, B., and Demaeyer,
J.: EUPP-benchmark/ESSD-Verification: version 1.0 release, Zenodo [code],
https://doi.org/10.5281/zenodo.7484371, 2023. a
Rabault, J., Müller, M., Voermans, J., Brazhnikov, D., Turnbull, I.,
Marchenko, A., Biuw, M., Nose, T., Waseda, T., Johansson, M., Breivik, Ø., Sutherland, G., Hole, L. R., Johnson, M., Jensen, A., Gundersen, O., Kristoffersen, Y., Babanin, A., Tedesco, P., Christensen, K. H., Kristiansen, M., Hope, G., Kodaira, T., de Aguiar, V., Taelman, C., Quigley, C. P., Filchuk, K., and Mahoney, A. R.: A
dataset of direct observations of sea ice drift and waves in ice, arXiv
[preprint], https://doi.org/10.48550/arXiv.2211.03565, 25 October 2022. a
Raftery, A. E., Gneiting, T., Balabdaoui, F., and Polakowski, M.: Using
Bayesian model averaging to calibrate forecast ensembles, Mon. Weather
Rev., 133, 1155–1174, 2005. a
Rasp, S., Dueben, P. D., Scher, S., Weyn, J. A., Mouatadid, S., and Thuerey,
N.: WeatherBench: A Benchmark Data Set for Data-Driven Weather Forecasting,
J. Adv. Model. Earth Sy., 12, e2020MS002203,
https://doi.org/10.1029/2020MS002203, 2020. a
Roberts, N., Ayliffe, B., Evans, G., Moseley, S., Rust, F., Sandford, C.,
Trzeciak, T., Abernethy, P., Beard, L., Crosswaite, N., Fitzpatrick, B.,
Flowerdew, J., Gale, T., Holly, L., Hopkinson, A., Hurst, K., Jackson, S.,
Jones, C., Mylne, K., Sampson, C., Sharpe, M., Wright, B., Backhouse, S.,
Baker, M., Brierley, D., Booton, A., Bysouth, C., Coulson, R., Coultas, S.,
Crocker, R., Harbord, R., Howard, K., Hughes, T., Mittermaier, M., Petch, J.,
Pillinger, T., Smart, V., Smith, E., and Worsfold, M.: IMPROVER: the new
probabilistic post processing system at the UK Met Office, B.
Am. Meteorol. Soc., 104, E680–E697, https://doi.org/10.1175/BAMS-D-21-0273.1, 2023. a
Taillardat, M., Mestre, O., Zamo, M., and Naveau, P.: Calibrated ensemble
forecasts using quantile regression forests and ensemble model output
statistics, Mon. Weather Rev., 144, 2375–2393, 2016. a
Vannitsem, S., Wilks, D. S., and Messner, J. W.: Statistical Postprocessing of
Ensemble Forecasts, 1st edn., edited by: Vannitsem, S., Wilks, D. S., and Messner, J. W., Elsevier, ISBN 978-0-12-812372-0, 2018. a
Vannitsem, S., Bremnes, J. B., Demaeyer, J., Evans, G. R., Flowerdew, J.,
Hemri, S., Lerch, S., Roberts, N., Theis, S., Atencia, A., Bouallègue,
Z. B., Bhend, J., Dabernig, M., Cruz, L. D., Hieta, L., Mestre, O., Moret,
L., Plenković, I. O., Schmeits, M., Taillardat, M., den Bergh, J. V.,
Schaeybroeck, B. V., Whan, K., and Ylhaisi, J.: Statistical Postprocessing
for Weather Forecasts: Review, Challenges, and Avenues in a Big Data World,
B. Am. Meteorol. Soc., 102, E681–E699,
https://doi.org/10.1175/BAMS-D-19-0308.1, 2021.
a, b
Van Schaeybroeck, B. and Vannitsem, S.: Ensemble post-processing using
member-by-member approaches: theoretical aspects, Q. J.
Roy. Meteor. Soc., 141, 807–818,
https://doi.org/10.1002/qj.2397, 2015. a
Wang, W., Yang, D., Hong, T., and Kleissl, J.: An archived dataset from the
ECMWF Ensemble Prediction System for probabilistic solar power forecasting,
Sol. Energy, 248, 64–75,
https://doi.org/10.1016/j.solener.2022.10.062, 2022. a
Watson-Parris, D., Rao, Y., Olivié, D., Seland, Ø., Nowack, P.,
Camps-Valls, G., Stier, P., Bouabid, S., Dewey, M., Fons, E., Gonzalez, J.,
Harder, P., Jeggle, K., Lenhardt, J., Manshausen, P., Novitasari, M., Ricard,
L., and Roesch, C.: ClimateBench v1.0: A Benchmark for Data-Driven Climate
Projections, J. Adv. Model. Earth Sy., 14,
e2021MS002954, https://doi.org/10.1029/2021MS002954, 2022. a
Wilks, D. S.: Multivariate ensemble Model Output Statistics using
empirical copulas, Q. J. Roy. Meteor. Soc.,
141, 945–952, https://doi.org/10.1002/qj.2414, 2015. a
Zandler, H., Haag, I., and Samimi, C.: Evaluation needs and temporal
performance differences of gridded precipitation products in peripheral
mountain regions, Scientific Reports, 9, 15118, https://doi.org/10.1038/s41598-019-51666-z, 2019. a
Zantedeschi, V., Falasca, F., Douglas, A., Strange, R., Kusner, M. J., and
Watson-Parris, D.: Cumulo: A dataset for learning cloud classes, arXiv
[preprint], https://doi.org/10.48550/arXiv.1911.04227, 5 November 2019. a
Zsótér, E.: Recent developments in extreme weather forecasting, ECMWF
Newsletter, 107, 8–17, 2006. a
Short summary
A benchmark dataset is proposed to compare different statistical postprocessing methods used in forecasting centers to properly calibrate ensemble weather forecasts. This dataset is based on ensemble forecasts covering a portion of central Europe and includes the corresponding observations. Examples on how to download and use the data are provided, a set of evaluation methods is proposed, and a first benchmark of several methods for the correction of 2 m temperature forecasts is performed.
A benchmark dataset is proposed to compare different statistical postprocessing methods used in...
Altmetrics
Final-revised paper
Preprint