the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
The EUPPBench postprocessing benchmark dataset v1.0
- 1Royal Meteorological Institute of Belgium, Brussels, Belgium
- 2European Meteorological Network (EUMETNET), Brussels, Belgium
- 3Federal Office of Meteorology and Climatology MeteoSwiss, Zurich, Switzerland
- 4Karlsruhe Institute of Technology, Karlsruhe, Germany
- 5Deutscher Wetterdienst, Offenbach, Germany
- 6GeoSphere Austria, Vienna, Austria
- 7European Centre for Medium-Range Weather Forecasts, Reading, United Kingdom
- 8Met Office, Exeter, United Kingdom
- 9University of Ljubljana, Faculty of Computer and Information Science, Slovenia
- 10University of Hildesheim, Hildesheim, Germany
- 11Slovenian Environment Agency, Ljubljana, Slovenia
- 12Bielefeld University, Bielefeld, Germany
- 13Meteo France, Ecole Nationale de la Meteorologie, Toulouse, France
- 1Royal Meteorological Institute of Belgium, Brussels, Belgium
- 2European Meteorological Network (EUMETNET), Brussels, Belgium
- 3Federal Office of Meteorology and Climatology MeteoSwiss, Zurich, Switzerland
- 4Karlsruhe Institute of Technology, Karlsruhe, Germany
- 5Deutscher Wetterdienst, Offenbach, Germany
- 6GeoSphere Austria, Vienna, Austria
- 7European Centre for Medium-Range Weather Forecasts, Reading, United Kingdom
- 8Met Office, Exeter, United Kingdom
- 9University of Ljubljana, Faculty of Computer and Information Science, Slovenia
- 10University of Hildesheim, Hildesheim, Germany
- 11Slovenian Environment Agency, Ljubljana, Slovenia
- 12Bielefeld University, Bielefeld, Germany
- 13Meteo France, Ecole Nationale de la Meteorologie, Toulouse, France
Abstract. Statistical Postprocessing of medium-range weather forecasts is an important component of modern forecasting systems. Since the beginning of modern data science, numerous new postprocessing methods have been proposed, complementing an already very diverse field. However, one of the questions that frequently arises when considering different methods in the framework of implementing operational postprocessing is the relative performance of the methods for a given specific task. It is particularly challenging to find or construct a common comprehensive dataset that can be used to perform such comparisons. Here, we introduce the first version of EUPPBench, a dataset of time-aligned forecasts and observations, with the aim to facilitate and standardize this process. This dataset is publicly available at https://github.com/EUPP-benchmark/climetlab-eumetnet-postprocessing-benchmark. We provide examples on how to download and use the data, propose a set of evaluation methods, and perform a first benchmark of several methods for the correction of 2-meter temperature forecasts.
- Preprint
(1346 KB) -
Supplement
(915 KB) - BibTeX
- EndNote
Jonathan Demaeyer et al.
Status: open (until 09 Mar 2023)
-
RC1: 'Comment on essd-2022-465', Anonymous Referee #1, 16 Jan 2023
reply
This paper introduces a benchmark dataset that can be used to compare the performance of statistical postprocessing methods for several key weather variables. Such a dataset is very useful to the postprocessing community in which numerous methods have been (and continue to be) developed but are often difficult to rank because they have been tested in different setups. While particular setups often require tailored solutions, at least subgroups of postprocessing methods can be considered to have interchangeable areas of application, and a benchmark data set can thus be useful to identify methods that stand out as particularly powerful. Along with the introduction of the data set the authors use the example of 2 metre temperatures to demonstrate how such an intercomparison of methods could be performed. I welcome this initiative, and I only have one minor comment and a few suggestions regarding language.
Minor comment:
Section 4.8: Is there a reference for this ANET method? If a reference is provided, the description of the method is adequate. Otherwise, more explanation is required for details like the dynamic attention mechanism. Since methodology is not the focus of this paper, such information could be provided as supplemental material.Typos and language:
60: complement -> complemented
113: as aforementioned -> as mentioned before
198: significant tests are run -> statistical tests are performed
216: validity -> valid
345: 'as' seems unnecessary here
Jonathan Demaeyer et al.
Data sets
EUPPBench postprocessing benchmark dataset - gridded data Demaeyer, Jonathan https://doi.org/10.5281/zenodo.7429236
EUPPBench postprocessing benchmark dataset - station data Bhend, Jonas; Dabernig, Markus; Demaeyer, Jonathan; Mestre, Olivier; Taillardat, Maxime https://doi.org/10.5281/zenodo.7428239
ESSD benchmark output data Chen, Jieyu; Dabernig, Markus; Demaeyer, Jonathan; Evans, Gavin; Faganeli Pucer, Jana; Hooper, Ben; Horat, Nina; Jobst, David; Lerch, Sebastian; Mlakar, Peter; Möller, Annette; Merše, Janko; Zied Ben Bouallègue https://doi.org/10.5281/zenodo.7469465
Model code and software
ESSD benchmark code Chen, Jieyu; Dabernig, Markus; Demaeyer, Jonathan; Evans, Gavin; Hooper, Ben; Horat, Nina; Jobst, David; Lerch, Sebastian; Mlakar, Peter; Möller, Annette; Zied Ben Bouallègue https://github.com/EUPP-benchmark/ESSD-benchmark
Jonathan Demaeyer et al.
Viewed
| HTML | XML | Total | Supplement | BibTeX | EndNote | |
|---|---|---|---|---|---|---|
| 283 | 125 | 9 | 417 | 20 | 9 | 6 |
- HTML: 283
- PDF: 125
- XML: 9
- Total: 417
- Supplement: 20
- BibTeX: 9
- EndNote: 6
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1