Articles | Volume 14, issue 11
Data description paper
22 Nov 2022
Data description paper |  | 22 Nov 2022

Harmonized gap-filled datasets from 20 urban flux tower sites

Mathew Lipson, Sue Grimmond, Martin Best, Winston T. L. Chow, Andreas Christen, Nektarios Chrysoulakis, Andrew Coutts, Ben Crawford, Stevan Earl, Jonathan Evans, Krzysztof Fortuniak, Bert G. Heusinkveld, Je-Woo Hong, Jinkyu Hong, Leena Järvi, Sungsoo Jo, Yeon-Hee Kim, Simone Kotthaus, Keunmin Lee, Valéry Masson, Joseph P. McFadden, Oliver Michels, Wlodzimierz Pawlak, Matthias Roth, Hirofumi Sugawara, Nigel Tapper, Erik Velasco, and Helen Claire Ward

A total of 20 urban neighbourhood-scale eddy covariance flux tower datasets are made openly available after being harmonized to create a 50 site–year collection with broad diversity in climate and urban surface characteristics. Variables needed as inputs for land surface models (incoming radiation, temperature, humidity, air pressure, wind and precipitation) are quality controlled, gap-filled and prepended with 10 years of reanalysis-derived local data, enabling an extended spin up to equilibrate models with local climate conditions. For both gap filling and spin up, ERA5 reanalysis meteorological data are bias corrected using tower-based observations, accounting for diurnal, seasonal and local urban effects not modelled in ERA5. The bias correction methods developed perform well compared to methods used in other datasets (e.g. WFDE5 or FLUXNET2015). Other variables (turbulent and upwelling radiation fluxes) are harmonized and quality controlled without gap filling. Site description metadata include local land cover fractions (buildings, roads, trees, grass etc.), building height and morphology, aerodynamic roughness estimates, population density and satellite imagery. This open collection can help extend our understanding of urban environmental processes through observational synthesis studies or in the evaluation of land surface environmental models in a wide range of urban settings. These data can be accessed from (Lipson et al., 2022).

1 Background

Tower mounted instruments allow the measurement of land–atmosphere fluxes (e.g. energy, momentum, water, carbon) and local meteorological conditions. These observations are one of the fundamental ways of improving both our understanding and ability to predict biogeophysical and weather-related processes at local scales. Regional and global networks of flux tower sites have helped extend our knowledge of ecosystem and climate science (Novick et al., 2018; Beringer et al., 2016; Yamamoto et al., 2005; Valentini, 2003). Over the last 25 years, networks such as FLUXNET have progressively increased access to flux data through open-source collections (Pastorello et al., 2020), extending the reach and impact of individual site observations through synthesis studies (Baldocchi, 2020) and multi-site environmental modelling and model evaluation projects (Best et al., 2015; Ukkola et al., 2022). However, with few urban sites included, urban areas have not benefited from the improved understanding or more extensive model evaluations that these collections can facilitate.

Urban areas are unique ecosystems, distinct from natural or rural landscapes. First, most people live in cities (UN, 2019) and infrastructure is concentrated within them. Therefore, climate-related health and economic impacts fall disproportionately within urban areas. Second, urban infrastructure (e.g. buildings and roads) along with transient human activities (e.g. energy consumption and irrigation) fundamentally alter surface energy, water and mass exchanges with the atmosphere, modifying local and larger-scale environmental conditions (Oke et al., 2017). Third, as built environments, urban areas are uniquely capable of actively mitigating and adapting to climate change.

Establishing and maintaining long-term flux sites in cities is particularly challenging because of the rarity of appropriate sites with homogenous fetch, the difficulty in gaining approval to access existing towers (e.g. for telecommunications), the cost of constructing tall towers over an aerodynamically rough surface and extremely limited long-term funding opportunities (Arnfield, 2003; Grimmond, 2006; Velasco and Roth, 2010; Feigenwinter et al., 2012; Grimmond and Ward, 2021). Thus, despite the diversity and importance of urban areas across the globe, urban flux tower data are relatively scarce, generally of short duration and rarely open source. Databases identifying urban observational programmes exist (e.g. the Urban Flux Network (Grimmond and Christen, 2012)), however urban flux tower datasets have not previously been brought together into a harmonized, gap-filled, open access collection.

We bring together quality-controlled data from 20 urban sites in an open collection that includes 50 observation years (Lipson et al., 2022). The sites are chosen to be diverse in both regional climates and urban characteristics. As evaluating land surface models is one key application for these data, we create continuous forcing datasets (i.e. with incoming radiation fluxes and other meteorological data) that are gap filled using site specific, bias corrected reanalysis data. Observations are also prepended with 10 years of site-specific reanalysis-derived meteorological data to allow modelled soil moisture and other conditions to equilibrate with local climate conditions during model spin up. These data can be used to drive land surface models offline at a single grid point. Other variables (turbulent and upwelling radiation fluxes) can be used to evaluative models in simulating land–atmosphere energy exchanges, or in observational synthesis studies.

Along with the meteorological data, site characteristics and metadata are provided in a common format. The metadata include tower location, land cover fractions, building heights and morphology, aerodynamic roughness parameter estimates, population density, estimated anthropogenic heat fluxes, site photos and satellite imagery. This collection can help extend our ability to model and understanding of environmental processes in different urban settings.

2 Methods

2.1 Site selection

The initial motivation for collating these flux tower and site data is for use in the Urban-PLUMBER multi-site model evaluation project, currently underway (Lipson, 2021). Urban-PLUMBER draws on methods from the first international urban land surface model comparison (Grimmond et al., 2010, 2011) and the Protocol for the Analysis of Land Surface Models Benchmarking Evaluation Project (PLUMBER (Best et al., 2015)). The latter evaluated land surface models in non-urban (vegetated) areas, while Urban-PLUMBER evaluates land surface models at 20 urban sites (Fig. 1, Table 1).

There is a two-fold use of these observational data in the model evaluation of Urban-PLUMBER:

  1. To provide local-scale meteorological input forcing to drive land surface models;

  2. To evaluate the performance of models, primarily assessing the local-scale exchange of radiant and turbulent heat fluxes between the surface and lower atmosphere.

With these objectives in mind, the following criteria are used to select flux tower sites:

  • appropriately sited for neighbourhood-scale conditions – i.e. within the inertial sub-layer, typically 2–5 times above the average building height and with relatively homogenous fetch (Grimmond, 2006; Barlow, 2014; Grimmond and Ward, 2021);

  • requested observations available at 30 or 60 min resolution (Table 2);

  • local site characteristics available for description and configuring models;

  • a preference for longer datasets (as this allows seasonal and inter-annual variability to be included);

  • collectively represent a diverse range of site characteristics and climates.

Figure 1Location of flux tower sites in this collection. Each site Köppen–Geiger climate classification (Beck et al., 2018) and the built land fraction around the tower are indicated at the bottom of the figure.

Table 1Site location and included observation (focus) period. Data providers may have longer observation periods available than are in this collection. Resolution is 30 min (or 60 min if denoted by *). All periods in universal time coordinated (UTC). US-Minneapolis data are split based on wind direction and fetch (Sect. 4.5).

Download Print Version | Download XLSX

Potential sites are identified from published site lists (Grimmond and Christen, 2012; Oke et al., 2017) and open calls for data (e.g. community newsletters (Lipson et al., 2020a), international conferences ((Lipson et al., 2020b, c) and social media professional networks). We deemed 20 sites sufficient for the evaluation project (Table 1), together covering 50 site–years. Included sites have built fractions (i.e. plan area fraction of all impervious surfaces including roofs, roads, other paving etc.) from 0.05 to 0.965, and are located in four major Köppen–Geiger (Beck et al., 2018) climate classes (Fig. 1). Eleven sites are in temperate climates, eight in cold (or continental) climates, and one in each of tropical and arid climates.

Sites are reasonably distributed across mean temperature and precipitation for global urban locations, but gaps remain, particularly in warm, wet and very cold climates (Fig. 2). Some urban flux observations in understudied regions were not included (e.g. Ouagadougou (Offerle et al., 2005), São Paulo (Ferreira et al., 2013), Guangzhou (Shi et al., 2019), Beijing (Dou et al., 2019)) because they do not meet the model evaluation project needs because of the relatively short observed periods for the available data. These regions and climates have large urban populations with significant environmental challenges and have few urban flux tower sites compared with Northern Hemisphere temperate or continental locations (Grimmond, 2006; Roth et al., 2017). Understudied regions and climates should be included in future collections when appropriate time series become available.

Figure 2Climatology of included sites compared with more than 70 000 global urban areas. Mean temperature and annual precipitation at the 20 tower sites (red, truncated site name, Table 1) from tower observations; global urban locations (grey) from ERA5 surface data (Hersbach et al., 2020, 2018) (2000–2010) from grid nearest to locations identified in the Global Rural–Urban Mapping Project (GRUMP) (Center for International Earth Science Information Network – CIESIN – Columbia University et al., 2017). Locations with rainfall above 3000 mm yr−1 (1.3 % of locations) and mean temperature below −3C (0.2  % of locations) are not shown.

2.2 Flux tower data

The observed data are provided in 30 or 60 min periods (Table 1), processed from high-frequency samples by individual observing groups. In the harmonized collection, timestamps are in coordinated universal time (UTC) indicating the end of the measurement period. Variables names and units use ALMA (Assistance for Land-surface Modelling Activities) conventions, a format used in previous land surface model comparisons.

Data are cleaned (Sect. 2.3: Quality control), forcing variables are gap filled (Sect. 2.4) and prepended with data derived from ERA5 (Sect. 2.5), after site-specific corrections (Sect. 2.6). An example of final prepended and gap filled data is shown in Fig. 3 for one site (UK-KingsCollege). Plots for all other variables and sites are also available in the collection (Lipson et al., 2022). Turbulent fluxes and upwelling radiation fluxes are not gap filled (Table 2).

Data are split into forcing and analysis variable sets (Table 2) to allow the forcing variables to be provided to modelling groups as input to run their models. The withheld analysis data are used by the coordinating group to assess the model outputs.

Some additional observed variables (Table 3) have, where practical, been included in the datasets after passing through the quality control steps. Missing forcing variables are obtained using bias corrected reanalysis data (Sect. 2.4). No gap filling is applied to analysis data or additional variables.

Figure 3Forcing time series with gap filling. Example shown for air temperature (Tair) at Kings College, London (UK-KingsCollege); (a) full forcing period, including 10 years of ERA5-derived data (red) prior to observations (black) used for model spin up, and (b) focus period used for analysis. Gaps are first filled from nearby tower measurements where available and short gaps (<=2 h) are linearly interpolated (blue). Remaining gaps are filled using the ERA5-derived time series which is seasonally and diurnally bias corrected using site observations. White lines show 7 d mean values. Similar plots are available for other sites within the site data collection (Lipson et al., 2022).

Table 2Forcing and analysis flux tower data variables. Short name description, units and positive direction use ALMA data conventions. Mean annual estimates of anthropogenic heat flux are included as site metadata. Analysis and additional data are not gap filled. Ground heat flux (Qg) is the heat flux into soil rather than total storage heat flux which is difficult to measure in urban areas (Grimmond and Oke, 1999).

Download Print Version | Download XLSX

Table 3Site climate classification missing and additional variables. Climate classification from Köppen–Geiger global dataset (Beck et al., 2018). Table 2 gives variable definitions.

* Note: as MX-Escandon LWdown data are unavailable during the 2011–2012 focus period, but were available in 2006, this earlier period is used to determine bias correction for ERA5 LWdown data.

Download Print Version | Download XLSX

2.3 Quality control and assurance

For each site the 30 or 60 min variables are calculated by data providers from high-frequency samples after applying their own quality control measures (e.g. Aubinet et al., 2012; Feigenwinter et al., 2012; Kotthaus and Grimmond, 2012; Vitale et al., 2020). The harmonized collection consists of the data retained after undergoing five additional quality control steps, in the following order:

  1. Out-of-range. Removal of unphysical values (e.g. negative shortwave radiation) using the ALMA expected range protocol (Bowling and Polcher, 2001).

  2. Night. Nocturnal shortwave radiation set to zero, based on civil twilight (when the sun is 6 below the horizon (Forsythe et al., 1995)).

  3. Constant. Four or more time steps with identical values (excluding zero values for shortwave radiation, rainfall and snowfall) are removed as suspicious.

  4. Outlier. Values outside ±4 standard deviations for each hour in a rolling 30 d window (to account for diurnal and seasonal variations) removed. Repeat with a larger tolerance (±5 SD, standard deviations) until no outliers remain (Schmid et al., 2000; Vickers and Mahrt, 1997). The outlier test is not applied to precipitation.

  5. Visual. Remaining suspect readings are removed manually via visual inspection.

These steps are undertaken in the processing script (see Sect. 5: Code availability), including periods identified through visual inspection (21 instances across all data). Data removed through quality control are indicated in plots of each variable at each site included in the data collection (Lipson et al., 2022). Note that quality control steps which eliminate observations at particular times (e.g. at night or after rainfall) can introduce biases (Grimmond, 2006). In addition, the outlier check values (step 4) are somewhat arbitrary (as noted in Vickers and Mahrt, 1997). Therefore, we also provide the “raw” observations (prior to quality control discussed here) in the collection as they may be more appropriate for some types of analyses.

Communication, or human errors, also have the potential to degrade or invalidate data (Menard et al., 2021). As part of quality assurance, project coordinators prepared an observational data protocol (Lipson et al., 2021) to explicitly set out requirements for data providers prior to submission of their data. The protocol documented instrument siting requirements, variables and data formats, dataset length and resolution, necessary site characteristic information and metadata, as well as the expectations for data handling, use and authorship. On receiving data, coordinators undertook further checks and identified errors that were not picked up by automated quality control. Identified errors included mislabelled variables and metadata, inconsistent timestamps and unit discrepancies. Many of the errors were identified by comparing provided data with secondary sources such as ERA5, nearby meteorological stations or previous publications. Errors were corrected collaboratively with data providers, some leading to corrections in primary data sources.

2.4 Gap filling

Three gap filling methods are used to create a continuous dataset for forcing variables, in the following order:

  • contemporaneous and nearby flux tower or weather observing sites (where available from data providers);

  • small gaps (≤2 h) are linearly interpolated from the adjoining observations;

  • larger gaps and a 10-year spin up period are filled with bias corrected ERA5 data (Sect. 2.6).

As only one site provided observed snowfall rate (JP-Yoyogi), ERA5 snowfall rates are used for all periods at other sites. At those sites the additional water equivalent from ERA5 snowfall is removed from subsequent observed rainfall until mass balance of observed total precipitation is achieved. This corrects melting snow being recorded as rainfall.

2.5 ERA5 reanalysis data

The ERA5 reanalysis product (Hersbach et al., 2020) assimilates global satellite, atmospheric and ground-based observations to constrain numerical weather prediction simulations, producing global output at 0.25 spatial and hourly temporal resolutions from 1959 to the present. It is therefore useful as a globally consistent and accessible source of meteorological data across space and time. ERA5, and its lower resolution predecessor ERA-Interim (Dee et al., 2011), have been used extensively to provide meteorological forcing data to drive land surface models and gap fill flux tower observations (Vuichard and Papale, 2015; Kokkonen et al., 2018; Pastorello et al., 2020; Ukkola et al., 2017, 2022).

The ERA5 hourly single level (Hersbach et al., 2018) dataset (retrieved from NCI Australia (Druken, 2020)) is used for gap filling missing observations within the focus periods (Table 1) and for the 10-year model spin up period. However, combining ERA5 data directly with urban flux tower observations has several deficiencies.

Grid-scale ERA5 data are not directly compatible with point-scale urban flux tower observations. This incompatibility is three-fold:

  1. Horizontally. The ERA5 grid cell area (of the order of 500 km2) does not match the flux footprint from tower observations (of order 1 km2). The ERA5 surface characteristics, including elevation, are based on an average description for the grid which may differ from surface characteristics around the observing tower, particularly in coastal or mountainous regions (Martens et al., 2020) in which many cities are located.

  2. Vertically. ERA5 provide near-surface variables (2 or 10 m above ground level), aligning with World Meteorological Organization (WMO) guidelines for standard regional observations taken over short grass (World Meteorological Organization, 2008). As the urban roughness elements (e.g. buildings) are much taller than grass, instruments are mounted on towers at heights greater than 2–5 times average building height in order to be located within the inertial sub layer or constant flux layer (Velasco and Roth, 2010; Barlow, 2014; Grimmond and Ward, 2021).

  3. Land surface. As the current operational ERA5 modelling systems do not include an urban land surface scheme (Boussetta et al., 2013; McNorton et al., 2021), other land types (grass, crops, shrubs, trees etc.) are used to characterize the grid cell (Table 4). Urban land surfaces are well known to alter local meteorological conditions (Oke et al., 2017), therefore ERA5 output will likely differ from locally observed conditions.

Outside of cities there are known diurnal and seasonal biases between the ERA5 near-surface variables and observations (Haiden et al., 2018; Betts et al., 2019; Nogueira, 2020; Martens et al., 2020; Jiang et al., 2021). These biases are an outcome of simplifying assumptions made in model parameterizations and inadequacies of modelling frameworks in general (Cucchi et al., 2020). Various approaches to reduce ERA5 biases in non-urban areas have been proposed. For example, the Water and Global Change (WATCH) Forcing Data (WFD) project use gridded observations to bias correct ERA-Interim data (Weedon et al., 2011), and more recently ERA5 data, creating the global WFDE5 dataset for impact studies (Cucchi et al., 2020). WFDE5 relies on the Climate Research Unit (CRU) monthly time series of gridded observations with resolution courser than ERA5 (New et al., 1999), requiring ERA5 to be regridded to a lower resolution. This may reduce the representativeness of the ERA5 data, particularly in heterogenous or complex terrain.

Alternatively, local observations can be used to bias correct ERA data, e.g. the linear regression corrections using tower observations applied to FLUXNET datasets (Vuichard and Papale, 2015; Pastorello et al., 2020). However, linear methods neither conserve the variability of observations (Vuichard and Papale, 2015) (Sect. 3), nor can they correct diurnal timing differences within ERA5 data (e.g. out of phase from urban temporal profiles, which are typically delayed compared with non-urban surfaces used in ERA5, Fig. 4).

To account for the mischaracterization of sites (Table 4) and other listed deficiencies in ERA5, we develop a novel set of methods to bias correct ERA5 data to better represent observed urban conditions (Sect. 2.6).

Table 4Surface cover information as specified in ERA5 differs from actual tower site characteristics (see Table 6), and so ERA5 data are corrected (Sect. 2.6). Given the ERA5 surface roughness values vary slightly through time, the values listed are indicative (from 1 January 2000). Effective roughness is our correction accounting for observed urban mean wind speeds.

Download Print Version | Download XLSX

2.6 Bias correction methods

Bias correction approaches used in the collection depend on the forcing variable (Table 2) and are described below.

2.7 Hourly and daily corrections

For incoming longwave radiation, air temperature, specific humidity and air pressure, the mean bias between ERA5 and local flux tower observations are calculated for each hour (h) and each day of a year (D) in a 60 d rolling window of a representative year (Fig. 4a). The calculated bias ηbias(D, h) is subtracted from the complete ERA5 time series ηERA5(t) to create a new corrected time series:

(1) η t = η ERA 5 t - η bias D , h .

The ERA5 data are from the grid nearest the observation site with at least 50 % land. The resulting corrected time series (e.g. Fig. 4b) is used for gap filling site observations and for the spin up period. The subroutine rolling_hourly_bias_correction in the file (Sect. 5) undertakes corrections with the following steps:

  1. If observations have a 30 min resolution, average to 60 min to match ERA5 periods;

  2. Remove ERA5 data for time periods where site observations are missing;

  3. Calculate mean for each hour with both data sets to create a “representative” year of 366 d;

  4. Extend to a 3-year period by duplication to provide smoother transitions at year end;

  5. Calculate hourly means in a 60 d rolling window across the repeating time series, excluding data in windows with less than 30 observations. Repeat mean calculation for greater smoothing;

  6. Calculate a time series of the bias between observed and ERA5 rolling means;

  7. Fill gaps in the bias time series by linear interpolation through each hour separately;

  8. Remove first and last year in the bias time series, using only the central year to bias correct each hour in the original ERA5 time series.

A 60 d rolling window is selected to smooth out individual weather events while still capturing seasonal variation. Repeating the representative year three times prior to smoothing ensures bias corrections match at the start and end of the year. The resulting set of bias correction curves (Fig. 4a) has greater robustness when multiple years are available.

Figure 4Urban-PLUMBER reanalysis bias correction methods. Demonstrated using air temperature (Tair) for the grid containing the King's College London site (UK-KingsCollege). (a) Hourly (colour) bias calculated for each day of a “representative” year and applied to entire ERA5 time series; (b) diurnal hourly mean Urban-PLUMBER correction (UP, red), observations (black), original ERA5 data (blue), WFDE5 bias corrected data (green) and linear bias correction method used in FLUXNET (LN, yellow). Our new UP method has smaller mean absolute errors (MAE) overall, and can correct both pattern and phase errors of ERA5 (Sect. 3).


2.7.1 Logarithmic wind profile correction

Wind speed differences between ERA5 and site observations can result from errors in modelled synoptic-scale speeds, differences in representative heights, and differences in surface aerodynamic properties like roughness and displacement height. To correct bias and maintain standard deviations of the wind components (U, V) of observations at sensor height (zsite), the following correction to ERA5 data is undertaken assuming both a logarithmic wind profile and neutral conditions (Goret et al., 2019):

(2) u corr = u ERA 5 ln z site - d site z 0 , site ln z grid - d grid z 0 , grid ,

where ucorr is the corrected wind speed at zsite. The ERA5 wind (uERA5) at 10 m (zgrid) is used with the site surface roughness (z0,site) and displacement height (dsite), and grid roughness (z0,grid) (Table 4) while assuming grid displacement height (dgrid) is zero for simplicity. If the resulting mean value of ucorr differs from observed mean value by more than 0.01 m s−1, then z0,grid is iteratively adapted until this threshold accuracy in mean wind speed is achieved. Derived z0,grid values are given in Table 4 (last column). Note this approach ignores seasonal effects from vegetation phenology and directional effects but ensures mean wind speeds are appropriate at the urban zsite while conserving variability within the ERA5 derived wind data.

2.7.2 Long-term precipitation correction

Total precipitation is an important variable in urban land surface models because of the effect on soil moisture which evolves over multi-year periods (Best and Grimmond, 2014). Most of the observational datasets included are not long enough to capture interannual variations. We therefore use longer term total precipitation (P) from nearby stations from the Global Historical Climatology Network – Daily (GHCND) (Menne et al., 2012) over a 10 year period to correct ERA5 rain and snow fluxes (ϕ) at each time step:

(3) ϕ corr = i = 1 10 yrs P GHCND i = 1 10 yrs P ERA 5 ϕ ERA 5 ( t ) .

Gaps in the nearest GHCND station data are progressively filled by the next nearest station until no gaps are present. If gaps could not be filled with GHNCD stations within 2 of latitude and longitude from the flux tower, and no alternative records are found (e.g. from national meteorological and hydrological services), then ERA5 rates are used unadjusted (viz., KR-Ochang and KR-Jungnang). This assumes precipitation occurs on the same dates and at the same times in ERA5 and observed datasets, which may become less valid under increasingly convective conditions.

2.7.3 Linear bias correction

The FLUXNET2015 collection of 212 flux tower sites (Pastorello et al., 2020) bias correction method uses linear regression between site observations and reanalysis data to derive one slope (s) and intercept (b) per site, hence “unbiasing” all ERA time steps (i):

(4) LN i = s ERA 5 i + b .

Following FLUXNET2015, global radiation and wind fields are assigned an intercept of zero, and precipitation is not linearly modified (Vuichard and Papale, 2015). FLUXNET2015 uses the coarser resolution ERA-Interim (spatial: 0.5 cf. 0.25; temporal 3 h cf. 1 h) than ERA5. In this evaluation we use ERA5, which is found to be a consistent improvement over ERA-Interim (Albergel et al., 2018). However, after assessment we chose not to use a linear method (LN) for correcting variables in this collection. Its description is retained here for comparison purposes (Sect. 3).

3 Gap filling evaluation

Site observations are quality controlled by individual data providers and collectively for this project (Sect. 2.3). The observed data required for forcing land surface models are then gap filled using a novel method of bias correcting reanalysis data. In this section, four methods which draw on ERA5 data are evaluated:

  1. ERA5. Nearest land-based 0.25 resolution ERA5 (Hersbach et al., 2018) grid without bias correction.

  2. W5. Nearest WFDE5 (Cucchi et al., 2020) grid (which uses bias correction from 0.5  CRU monthly gridded observations).

  3. UP. The Urban-PLUMBER methods described here (using site observations for bias correction).

  4. LN. Linear methods based on FLUXNET2015 (Vuichard and Papale, 2015; Pastorello et al., 2020) (using site observations for bias correction).

To evaluate the methods available for gap filling, quality-controlled tower site observations (Oi) are used to assess the calculated value (η) at timestep i using three metrics:

  • a.

    Mean bias error (MBE): i=1nηi-Oin

  • b.

    Mean absolute error (MAE): i=1nηi-Oin

  • c.

    Normalized standard deviation (nSD): i=1nηi-ηn-12/i=1nOi-On-12.

Where O and η are time-averaged over n data points. The timestamps for each variable are made consistent between the data sources: ERA5 are hourly time ending or instantaneous data (Hersbach et al., 2018); 60 min site observations are hour ending; 30 min site observations are converted to 60 min time ending by averaging; whereas as the WFDE5 SWdown, LWdown and Rainf are natively 60 min time beginning (Cucchi et al., 2020), they are shifted forward to match the time ending timestamps; and WFDE5 Tair, Qair, Psurf and Wind are instantaneous samples on the hour so their timestamp remains unchanged.

To summarize each metric for the 20 sites, we use boxplots (Fig. 5) for the seven forcing variables. The evaluation inherently considers the net differences associated with both the spatial (vertical and surface cover) differences and errors (model and observation) from the two datasets. Uncorrected ERA5 (blue, Fig. 5) biases are generally negative for Tair, LWdown and Wind, and generally positive for Qair and SWdown. These biases can be partly explained by the ERA5 framework not including an urban surface model. For example, the well-documented warmer air temperature in cities (urban heat island) are not modelled in ERA5 because natural land surfaces are assumed in simulations (Table 4), although ERA5 can include an urban signal if the data assimilated are from within urban areas (Tang et al., 2021). Qair shows a general positive bias as evapotranspiration will be overestimated in ERA5 without an urban land surface representation. Likewise, ERA5 SWdown are overestimated and LWdown underestimated possibly because urban air pollution effects are not included (Oke, 1988).

Other discrepancies between ERA5 data and site data arise from elevation differences in height above sea level (a.s.l.) of the ERA5 grid and site. For example, the MX-Escandon tower in Mexico City measurement height is 2277 m a.s.l., whereas the ERA5 grid cell is assigned a surface elevation of 2540 m a.s.l. because the cell includes nearby mountains. This 263 m difference causes a negative bias to Psurf of 2594 Pa and contributes to a Tair difference of −2.15 K. Additionally, orographic uplift increases the grid cell rainfall, leading to positive ERA5 bias of 2.25 × 10−5 kg m−2 s−1 (+710 mm yr−1) compared with the MX-Escandon site observations. The ERA5 rainfall bias is even more pronounced at CA-Sunset in Vancouver, Canada, with a +1178 mm yr−1 bias. These results are consistent with other studies highlighting discrepancies between reanalysis and local data in mountainous regions (Kokkonen et al., 2018).

The other three methods (WFDE5 (W5), linear debiasing (LN) and the Urban-PLUMBER corrections (UP)) apply bias corrections to ERA5 data, and so may be expected to reduce ERA5 errors. However, W5 does not reduce errors at these sites, most likely because the observations used for W5 bias correction are at very different spatial scales to the flux tower footprints (2500 km2 versus 1 km2) and include observations from non-urban locations. The LN and UP methods eliminate spatial mismatches by drawing on local site or nearby rain gauge observations. As such, they do reduce the mean bias error to near zero for most variables. Notably, the UP methods outperform LN methods in normalized standard deviation. As Vuichard and Papale (2015) noted, linear methods do not conserve the variability of observations, nor can they correct for diurnal phase shifts of some variables observed in urban areas (Fig. 4). Therefore, we consider the UP methods to be the most appropriate at these urban sites. However, we apply no bias corrections to SWdown because the hourly and daily corrections (i.e. UP methods applied to other variables) adversely impact the standard deviation errors, as does the LN method for SWdown.

Figure 5Evaluation of bias correction methods. Four methods (colour) to create gap filled observed time series data: ERA5 (blue), WFDE5 (W5, green), linear debiasing (LN, orange), UP (red, this study) using (row 1) mean bias error, (row 2) mean absolute error, (row 3) normalized standard deviation, with the 20 individual sites (dots), and ideal agreement with observations (red line) and boxplot showing distribution. The UP corrections (selected for use in this study) have lower overall errors (cf. other methods) except SWdown, where no corrections to ERA5 are applied.


4 Data availability

Data described in this manuscript can be accessed from (Lipson et al., 2022) under a Creative Commons Attribution licence (CC-BY-4.0).

We recommend data users consult with site contributing authors and/or the coordination team early (i.e. planning stage) in projects that plan to use these data. Relevant contacts are included in site metadata.

5 Code availability

Code used to process datasets are available at (Lipson, 2022a).

Code used to create paper figures are available at (Lipson, 2022b).

6 Data records

6.1 Data format

Time series data and site descriptive metadata are recorded in both plain text and netCDF4 (Rew et al., 1989) formats. Each site folder contains the following time series:

  • [sitename]_raw_observations_[version]: site observed before project-wide quality control and gap filling (Table 1 gives period)

  • [sitename]_clean_observations_[version]: after project-wide quality control and gap filling (Table 1 gives period)

  • [sitename]_metforcing_[version]: continuous observations with reanalysis-derived data after quality control and gap filling (Table 2; forcing data for 10-year spin up, then Table 1 periods)

  • [sitename]_era5_corrected_[version]: continuous time series (1990–2020) of bias corrected ERA5 reanalysis meteorological data (as used for gap filling and prepending metforcing observations).

Each site folder also contains the following site metadata:

  • [sitename]_sitedata_[version].csv: comma separated text file for site characteristics metadata e.g. latitude, longitude, surface cover fraction and morphology (Tables 5, 6). This site characteristic data are also included within the metforcing netcdf (for convenience)

  • index.html: a summary page of site information in html format, including site characteristics, site images, time series, gap filling, quality control and diurnal plots.

6.2 Time series metadata

The time series files include the following metadata:

  • title: short description of the file

  • summary: longer description of the file

  • sitename: site code (e.g. AU-Preston)

  • long_sitename: site long name, including city and country information

  • version: version of current file

  • time_coverage_start: start of time series in UTC (includes spin up) with period ending timestamps

  • time_coverage_end: end of time series in UTC with period ending timestamps

  • time_analysis_start: start of observed (focus) period in UTC with period ending timestamps

  • time_shown_in: time standard (always UTC)

  • local_utc_offset_hours: offset in hours of local time from UTC

  • timestep_interval_seconds: period of block averaging in seconds (time step)

  • timestep_number_spinup: number of time steps prior to observed focus period

  • timestep_number_analysis: number of time steps in observed focus period

  • project_contact: contact details for the Urban-PLUMBER project coordinators

  • observations_contact: contact details of the observational site data providers

  • observations_reference: published references associated with the observations

  • date_created: date and time of creation of this file

  • source: repository for processing code

  • comment: additional comments associated with this dataset (e.g. excluded wind sectors).

Text file time series include metadata headers indicated with a hash (#) at line beginnings. Columns are headed by variable names in ALMA format (Table 2). NetCDF4 files include identical data, with additional attributes for each variable:

  • long_name: plain language description of variable

  • standard_name: equivalent variable name under the CF (climate and forecast) conventions

  • units: SI (international system) units

  • ancillary_variables: name of the associated quality control flag variable.

NetCDF files also include site characteristics parameter values and descriptions (Table 5). Times in all datasets are UTC. Python programmes are provided (Lipson, 2022a) to convert UTC times to local standard time (, and netCDF to text (

Table 5Site characteristic metadata description and units. Parameters are determined for the turbulent flux footprint extent (Table 6), except 1–4 which are applicable to the tower itself, and 19 which is a function of the radiometer field of view (Offerle et al., 2003) and differs from the turbulent flux footprint (Schmid et al., 1991).

Download Print Version | Download XLSX

6.3 Site characteristics metadata

Site characteristics (Tables 5, 6) are essential for any use of these data, and fundamental to application of land surface models. These metadata are provided in two machine readable forms (plain text in csv files and netCDF4). The metadata are primarily drawn from published sources or as advised by the data providers. If local parameters are not known, values are estimated from high-resolution global datasets or derived from empirical relations. The sources for each parameter are included within the site characteristic metadata.

There are numerous methods to estimate the probable extent and weighting of turbulent fluxes footprints relative to the eddy covariance sensors located on a flux tower (Velasco and Roth, 2010). The eddy covariance flux footprint provides a basis to identify which area (and weighting) should be used to estimate the land surface fractions impacting the measurements. Some studies in this collection determine the footprint and resulting land cover fractions dynamically (e.g. for each 30 min period based on that period's observed atmospheric variables such as stability and wind direction), whereas others used a constant radius (e.g. based on the footprint climatology or rule of thumb) (Table 6). Standardizing the method to determine land cover fractions across sites is beyond the scope of this work, so users of metadata should be mindful of these differences.

Different methods for estimating surface roughness length and zero-plane displacement height can give significantly different values (Kent et al., 2017). Given that different data providers have derived values using different methods, we also provide values using two consistent morphometric methods (Macdonald (Macdonald et al., 1998) and Kanda (Kanda et al., 2013); Table 5, parameters 26–29) derived from surface fraction and building height parameters within the measurement footprint (Table 6). The Kanda modification to the Macdonald method accounts for the variability in roughness element height, resulting in larger displacement heights which are closer to estimates made with anemometric methods (Kent et al., 2017). The Macdonald method assumes that all the buildings have the same average roughness element height. However, care must be taken when using Kanda values as some urban land surface models expect displacement height to be always lower than average building height (Hertwig et al., 2020).

Where not known, building height standard deviation (σH) is estimated from an empirical relation to building mean height (Have) (Kanda et al., 2013):

(5) σ H = 1.05 H ave - 3.7 .

Similarly, unknown local wall to plan area ratios (λw) are derived from roof area fraction (λp) and canyon height to width ratio (H/W) assuming an infinite canyon geometry (Masson et al., 2020):

(6) λ w = 2 1 - λ p H / W .

Frontal area index (λf) is sometimes reported in site literature without H/W or λw, in which case these are estimated (again assuming an infinite canyon geometry) with (Porson et al., 2010):

(7) λ f = 2 π 1 - λ p H / W .

Where not known or provided, mean annual anthropogenic heat flux (Varquez et al., 2021) or soil characteristics (Hengl, 2018a, b, c) are estimated from global datasets at 1 km or lower resolutions.

Table 6Select site characteristic values (see Table 5 for definitions). Other site characteristic values and sources are provided within the collection (Lipson et al., 2022). Areas analysed for land cover fractions and roughness parameters are based on either a static radius around the flux tower (value given) or a dynamic footprint model (fpm). For the latter, the spatial extents are the order of a few hundred metres but are dynamic varying for example with atmospheric stability and wind direction (Grimmond and Ward, 2021).

Download Print Version | Download XLSX

Table 7Site wind sector exclusions. Sites with sensible and latent heat fluxes excluded because of land cover or land use differences by wind sectors as described in the reference provided. Maps of these sectors are provided in the site data collection (Lipson et al., 2022).

Download Print Version | Download XLSX

Table 8Funding acknowledgements for individual sites.

Download Print Version | Download XLSX

6.4 Data flags

Each variable for each time step has a quality control (qc) flag. For example, LWdown_qc lists qc flags for LWdown at each time step. Flag numbers are consistent across all variables:

  • 0.

    observed by measurement at site and passes project quality control tests

  • 1.

    filled by observation; interpolated from site observations over short (2 h) periods OR filled by observations from nearby (<10 km) stations over longer periods

  • 2.

    filled by ERA5; derived from ERA5 with site specific bias correction

  • 3.

    missing or removed through quality control (occurs only in time series without gap filling).

6.5 Wind sector exclusions

Turbulent flux data are excluded from certain wind directions (Table 7) because of

  • interference on flow from tower structure (as identified by data providers);

  • markedly different land cover characteristics from sectors of interest (with guidance from data providers).

The US-Minneapolis site has different surface cover by wind direction but is retained in the collection because of both a long observation period and its distinct land cover characteristics. Following previous studies (Menzer and McFadden, 2017) we subdivide these data into low density residential area (northern sectors, US-Minneapolis2) and irrigated grassland with few built structures (south, US-Minneapolis1). Each are given their own site time series and metadata, resulting in 21 datasets.

Author contributions

ML, SG and MB conceived and coordinated the project, and prepared the protocols for contributing authors. ML collated site datasets, wrote processing code, developed and undertook analysis of bias correction methods, prepared figures, prepared datasets and drafted the paper with guidance from SG and MB. All other authors (listed alphabetically) collected primary data, prepared site information, processed datasets for inclusion in the collection and contributed to the paper. Table 8 lists contributing author site affiliation.

Competing interests

The contact author has declared that none of the authors has any competing interests.


Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.


We would like to thank the vast number of people involved in day-to-day running of these sites that have been involved in instrument and tower installations (permitting, purchasing and site installation), routine (and unexpected event) maintenance, data collection, routine data processing and final data processing. We thank all those that have provided sites for the towers to be located and sometimes power and internet access. We acknowledge the essential funding for the instrumentation and other infrastructure, for staff (administrative, technical and scientific) and students for these activities. We also thank those who offered data for use in this project which are not included at this time.

Financial support

The project coordinating team has been supported by UNSW Sydney and the Australian Research Council (ARC) Centre of Excellence for Climate System Science (grant no. CE110001028), University of Reading, the Met Office and ERC urbisphere 855005. Computation support has been provided by the ARC Centre of Excellence for Climate Extremes (grant no. CE170100023) and National Computational Infrastructure (NCI) Australia. It contains modified Copernicus Climate Change Service Information. Site-affiliated acknowledgements are listed in Table 8.

Review statement

This paper was edited by Hanqin Tian and reviewed by two anonymous referees.


Albergel, C., Dutra, E., Munier, S., Calvet, J.-C., Munoz-Sabater, J., de Rosnay, P., and Balsamo, G.: ERA-5 and ERA-Interim driven ISBA land surface model simulations: which one performs better?, Hydrol. Earth Syst. Sci., 22, 3515–3532,, 2018. 

Arnfield, A. J.: Two decades of urban climate research: a review of turbulence, exchanges of energy and water, and the urban heat island, Int. J. Climatol., 23, 1–26,, 2003. 

Aubinet, M., Vesala, T., and Papale, D.: Eddy Covariance: A Practical Guide to Measurement and Data Analysis, 1st edn., edited by: Aubinet, M., Vesala, T., and Papale, D., Springer Science & Business Media, 451 pp., ISBN 978-94-007-9630-0, 2012. 

Baldocchi, D. D.: How eddy covariance flux measurements have contributed to our understanding of Global Change Biology, Glob. Change Biol., 26, 242–260,, 2020. 

Barlow, J. F.: Progress in observing and modelling the urban boundary layer, Urban Climate, 10, Part 2, 216–240,, 2014. 

Beck, H. E., Zimmermann, N. E., McVicar, T. R., Vergopolan, N., Berg, A., and Wood, E. F.: Present and future Köppen-Geiger climate classification maps at 1-km resolution, Scientific Data, 5, 180214,, 2018. 

Beringer, J., Hutley, L. B., McHugh, I., Arndt, S. K., Campbell, D., Cleugh, H. A., Cleverly, J., Resco de Dios, V., Eamus, D., Evans, B., Ewenz, C., Grace, P., Griebel, A., Haverd, V., Hinko-Najera, N., Huete, A., Isaac, P., Kanniah, K., Leuning, R., Liddell, M. J., Macfarlane, C., Meyer, W., Moore, C., Pendall, E., Phillips, A., Phillips, R. L., Prober, S. M., Restrepo-Coupe, N., Rutledge, S., Schroder, I., Silberstein, R., Southall, P., Yee, M. S., Tapper, N. J., van Gorsel, E., Vote, C., Walker, J., and Wardlaw, T.: An introduction to the Australian and New Zealand flux tower network – OzFlux, Biogeosciences, 13, 5895–5916,, 2016. 

Best, M. J. and Grimmond, C. S. B.: Importance of initial state and atmospheric conditions for urban land surface models' performance, Urban Climate, 10, 387–406,, 2014. 

Best, M. J., Abramowitz, G., Johnson, H. R., Pitman, A. J., Balsamo, G., Boone, A., Cuntz, M., Decharme, B., Dirmeyer, P. A., Dong, J., Ek, M., Guo, Z., Haverd, V., Hurk, B. J. J. van den, Nearing, G. S., Pak, B., Peters-Lidard, C., Santanello, J. A., Stevens, L., and Vuichard, N.: The Plumbing of Land Surface Models: Benchmarking Model Performance, J. Hydrometeorol., 16, 1425–1442,, 2015. 

Betts, A. K., Chan, D. Z., and Desjardins, R. L.: Near-Surface Biases in ERA5 Over the Canadian Prairies, Front. Environ. Sci., 7,, 2019. 

Bjorkegren, A. B., Grimmond, C. S. B., Kotthaus, S., and Malamud, B. D.: CO2 emission estimation in the urban environment: Measurement of the CO2 storage term, Atmos. Environ., 122, 775–790,, 2015. 

Boussetta, S., Balsamo, G., Beljaars, A., Panareda, A.-A., Calvet, J.-C., Jacobs, C., Hurk, B. van den, Viterbo, P., Lafont, S., Dutra, E., Jarlan, L., Balzarolo, M., Papale, D., and Werf, G. van der: Natural land carbon dioxide exchanges in the ECMWF integrated forecasting system: Implementation and offline validation, J. Geophys. Res.-Atmos., 118, 5923–5946,, 2013. 

Bowling, L. and Polcher, J.: The ALMA data exchange convention, (last access: 3 June 2021), 2001. 

Center for International Earth Science Information Network – CIESIN – Columbia University, International Food Policy Research Institute – IFPRI, The World Bank, and Centro Internacional de Agricultura Tropical – CIAT: Global Rural-Urban Mapping Project, Version 1 (GRUMPv1): Settlement Points, Revision 01, NASA Socioeconomic Data and Applications Center (SEDAC) [data set], Palisades, NY,, 2017. 

Chow, W.: Eddy covariance data measured at the CAP LTER flux tower located in the west Phoenix, AZ neighborhood of Maryvale from 2011-12-16 through 2012-12-31, EDI Data Portal [data set],, 2017. 

Chow, W. T. L., Volo, T. J., Vivoni, E. R., Jenerette, G. D., and Ruddell, B. L.: Seasonal dynamics of a suburban energy balance in Phoenix, Arizona, Int. J. Climatol., 34, 3863–3880,, 2014. 

Christen, A., Coops, N. C., Crawford, B. R., Kellett, R., Liss, K. N., Olchovski, I., Tooke, T. R., van der Laan, M., and Voogt, J. A.: Validation of modeled carbon-dioxide emissions from an urban neighborhood with direct eddy-covariance measurements, Atmos. Environ., 45, 6057–6069,, 2011. 

Coutts, A. M., Beringer, J., and Tapper, N. J.: Characteristics influencing the variability of urban CO2 fluxes in Melbourne, Australia, Atmos. Environ., 41, 51–62,, 2007a. 

Coutts, A. M., Beringer, J., and Tapper, N. J.: Impact of Increasing Urban Density on Local Climate: Spatial and Temporal Variations in the Surface Energy Balance in Melbourne, Australia, J. Appl. Meteor. Climatol., 46, 477–493,, 2007b. 

Crawford, B. and Christen, A.: Spatial source attribution of measured urban eddy covariance CO2 fluxes, Theor. Appl. Climatol., 119, 733–755,, 2015. 

Crawford, B., Grimmond, C. S. B., and Christen, A.: Five years of carbon dioxide fluxes measurements in a highly vegetated suburban area, Atmos. Environ., 45, 896–905,, 2011. 

Cucchi, M., Weedon, G. P., Amici, A., Bellouin, N., Lange, S., Müller Schmied, H., Hersbach, H., and Buontempo, C.: WFDE5: bias-adjusted ERA5 reanalysis data for impact studies, Earth Syst. Sci. Data, 12, 2097–2120,, 2020. 

Dee, D. P., Uppala, S. M., Simmons, A. J., Berrisford, P., Poli, P., Kobayashi, S., Andrae, U., Balmaseda, M. A., Balsamo, G., Bauer, P., Bechtold, P., Beljaars, A. C. M., van de Berg, L., Bidlot, J., Bormann, N., Delsol, C., Dragani, R., Fuentes, M., Geer, A. J., Haimberger, L., Healy, S. B., Hersbach, H., Hólm, E. V., Isaksen, L., Kållberg, P., Köhler, M., Matricardi, M., McNally, A. P., Monge-Sanz, B. M., Morcrette, J.-J., Park, B.-K., Peubey, C., de Rosnay, P., Tavolato, C., Thépaut, J.-N., and Vitart, F.: The ERA-Interim reanalysis: configuration and performance of the data assimilation system, Q. J. Roy. Meteor. Soc., 137, 553–597,, 2011. 

Dou, J., Grimmond, S., Cheng, Z., Miao, S., Feng, D., and Liao, M.: Summertime surface energy balance fluxes at two Beijing sites, Int. J. Climatol., 39, 2793–2810,, 2019. 

Druken, K.: ERA5 Replicated Datasets, NCI Data Catalogue [data set],, 2020. 

Feigenwinter, C., Vogt, R., and Christen, A.: Eddy Covariance Measurements Over Urban Areas, in: Eddy Covariance: A Practical Guide to Measurement and Data Analysis, edited by: Aubinet, M., Vesala, T., and Papale, D., Springer Netherlands, Dordrecht, 377–397,, 2012. 

Ferreira, M. J., de Oliveira, A. P., and Soares, J.: Diurnal variation in stored energy flux in São Paulo city, Brazil, Urban Climate, 5, 36–51,, 2013. 

Forsythe, W. C., Rykiel, E. J., Stahl, R. S., Wu, H., and Schoolfield, R. M.: A model comparison for daylength as a function of latitude and day of year, Ecol. Model., 80, 87–95,, 1995. 

Fortuniak, K., Kłysik, K., and Siedlecki, M.: New measurements of the energy balance components in Łódź, in: Preprints, sixth International Conference on Urban Climate, Göteborg, Sweden, Sixth International Conference On Urban Climate, Göteborg, Sweden, 12–16 June, 2006, 64–67, (last access: 14 November 2022), 2006. 

Fortuniak, K., Pawlak, W., and Siedlecki, M.: Integral Turbulence Statistics Over a Central European City Centre, Bound.-Lay. Meteorol., 146, 257–276,, 2013. 

Goret, M., Masson, V., Schoetter, R., and Moine, M.-P.: Inclusion of CO2 flux modelling in an urban canopy layer model and an evaluation over an old European city centre, Atmospheric Environment: X, 3, 100042,, 2019. 

Grimmond, C. S. B.: Progress in measuring and observing the urban atmosphere, Theor. Appl. Climatol., 84, 3–22,, 2006. 

Grimmond, C. S. B. and Oke, T. R.: Heat Storage in Urban Areas: Local-Scale Observations and Evaluation of a Simple Model, J. Appl. Meteor., 38, 922–940,<0922:HSIUAL>2.0.CO;2, 1999. 

Grimmond, C. S. B., Blackett, M., Best, M. J., Barlow, J., Baik, J.-J., Belcher, S. E., Bohnenstengel, S. I., Calmet, I., Chen, F., Dandou, A., Fortuniak, K., Gouvea, M. L., Hamdi, R., Hendry, M., Kawai, T., Kawamoto, Y., Kondo, H., Krayenhoff, E. S., Lee, S.-H., and Loridan, T.: The International Urban Energy Balance Models Comparison Project: First Results from Phase 1, J. Appl. Meteorol. Clim., 49, 1268–1292,, 2010. 

Grimmond, C. S. B., Blackett, M., Best, M. J., Baik, J.-J., Belcher, S. E., Beringer, J., Bohnenstengel, S. I., Calmet, I., Chen, F., Coutts, A., Dandou, A., Fortuniak, K., Gouvea, M. L., Hamdi, R., Hendry, M., Kanda, M., Kawai, T., Kawamoto, Y., Kondo, H., Krayenhoff, E. S., Lee, S.-H., Loridan, T., Martilli, A., Masson, V., Miao, S., Oleson, K., Ooka, R., Pigeon, G., Porson, A., Ryu, Y.-H., Salamanca, F., Steeneveld, G. J., Tombrou, M., Voogt, J. A., Young, D. T., and Zhang, N.: Initial results from Phase 2 of the international urban energy balance model comparison, Int. J. Climatol., 31, 244–272,, 2011. 

Grimmond, S. and Christen, A.: Flux measurements in urban ecosystems, FluxLetter – Newsletter of Fluxnet, 5, 1–8, (last access: 14 November 2022), 2012. 

Grimmond, S. and Ward, H. C.: Urban Measurements and Their Interpretation, in: Springer Handbook of Atmospheric Measurements, edited by: Foken, T., Springer International Publishing, Cham, 1407–1437,, 2021. 

Haiden, T., Sandu, I., Balsamo, G., Arduini, G., and Beljaars, A.: Addressing biases in near-surface forecasts, ECMWF Newsletter, 157, 20–25, 2018. 

Hengl, T.: Clay content in % (kg/kg) at 6 standard depths (0, 10, 30, 60, 100 and 200 cm) at 250 m resolution (v0.2), Zenodo [data set],, 2018a. 

Hengl, T.: Sand content in % (kg/kg) at 6 standard depths (0, 10, 30, 60, 100 and 200 cm) at 250 m resolution (v0.2), Zenodo [data set],, 2018b. 

Hengl, T.: Soil bulk density (fine earth) 10 × kg/m-cubic at 6 standard depths (0, 10, 30, 60, 100 and 200 cm) at 250 m resolution (v0.2), Zenodo [data set],, 2018c. 

Hersbach, H., Bell, B., Berrisford, P., Biavati, G., Horányi, A., Muñoz Sabater, J., Nicolas, J., Peubey, C., Radu, R., Rozum, I., and others: ERA5 hourly data on single levels from 1979 to present, Copernicus Climate Change Service (C3S) Climate Data Store (CDS) [data set],, 2018. 

Hersbach, H., Bell, B., Berrisford, P., Hirahara, S., Horányi, A., Muñoz-Sabater, J., Nicolas, J., Peubey, C., Radu, R., Schepers, D., Simmons, A., Soci, C., Abdalla, S., Abellan, X., Balsamo, G., Bechtold, P., Biavati, G., Bidlot, J., Bonavita, M., Chiara, G. D., Dahlgren, P., Dee, D., Diamantakis, M., Dragani, R., Flemming, J., Forbes, R., Fuentes, M., Geer, A., Haimberger, L., Healy, S., Hogan, R. J., Hólm, E., Janisková, M., Keeley, S., Laloyaux, P., Lopez, P., Lupu, C., Radnoti, G., Rosnay, P. de, Rozum, I., Vamborg, F., Villaume, S., and Thépaut, J.-N.: The ERA5 global reanalysis, Q. J. Roy. Meteor. Soc., 146, 1999–2049,, 2020. 

Hertwig, D., Grimmond, S., Hendry, M. A., Saunders, B., Wang, Z., Jeoffrion, M., Vidale, P. L., McGuire, P. C., Bohnenstengel, S. I., Ward, H. C., and Kotthaus, S.: Urban signals in high-resolution weather and climate simulations: role of urban land-surface characterisation, Theor. Appl. Climatol., 142, 701–728,, 2020. 

Hirano, T., Sugawara, H., Murayama, S., and Kondo, H.: Diurnal Variation of CO2 Flux in an Urban Area of Tokyo, Sola, 11, 100–103,, 2015. 

Hong, J., Lee, K., and Hong, J.-W.: Observational data of Ochang and Jungnang in Korea, EAPL at Yonsei University [data set],, 2020. 

Hong, J.-W., Hong, J., Chun, J., Lee, Y. H., Chang, L.-S., Lee, J.-B., Yi, K., Park, Y.-S., Byun, Y.-H., and Joo, S.: Comparative assessment of net CO2 exchange across an urbanization gradient in Korea based on eddy covariance measurements, Carbon Balance and Management, 14, 13,, 2019. 

Hong, S., Kim, J., Byun, Y., Hong, J., Hong., J., Lee, K., Y. Park, Lee, S., and Kim, Y.: Intra-urban variations of the CO2 fluxes at the surface-atmosphere interface in the Seoul metropolitan area, Carbon Balance and Management, in revision, 2022. 

Ishidoya, S., Sugawara, H., Terao, Y., Kaneyasu, N., Aoki, N., Tsuboi, K., and Kondo, H.: O2: CO2 exchange ratio for net turbulent flux observed in an urban area of Tokyo, Japan, and its application to an evaluation of anthropogenic CO2 emissions, Atmos. Chem. Phys., 20, 5293–5308,, 2020. 

Järvi, L., Rannik, Ü., Kokkonen, T. V., Kurppa, M., Karppinen, A., Kouznetsov, R. D., Rantala, P., Vesala, T., and Wood, C. R.: Uncertainty of eddy covariance flux measurements over an urban area based on two towers, Atmos. Meas. Tech., 11, 5421–5438,, 2018. 

Jiang, Q., Li, W., Fan, Z., He, X., Sun, W., Chen, S., Wen, J., Gao, J., and Wang, J.: Evaluation of the ERA5 reanalysis precipitation dataset over Chinese Mainland, J. Hydrol., 595, 125660,, 2021. 

Kanda, M., Inagaki, A., Miyamoto, T., Gryschka, M., and Raasch, S.: A New Aerodynamic Parametrization for Real Urban Surfaces, Bound.-Lay. Meteorol., 148, 357–377,, 2013. 

Karsisto, P., Fortelius, C., Demuzere, M., Grimmond, C. S. B., W., O. K., Kouznetsov, R., Masson, V., and Järvi, L.: Seasonal surface urban energy balance and wintertime stability simulated using three land-surface models in the high-latitude city Helsinki, Q. J. Roy. Meteor. Soc., 142, 401–417,, 2016. 

Kent, C. W., Grimmond, S., Barlow, J., Gatey, D., Kotthaus, S., Lindberg, F., and Halios, C. H.: Evaluation of Urban Local-Scale Aerodynamic Parameters: Implications for the Vertical Profile of Wind Speed and for Source Areas, Bound.-Lay. Meteorol., 164, 183–213,, 2017. 

Kokkonen, T. V., Grimmond, C. S. B., Räty, O., Ward, H. C., Christen, A., Oke, T. R., Kotthaus, S., and Järvi, L.: Sensitivity of Surface Urban Energy and Water Balance Scheme (SUEWS) to downscaling of reanalysis forcing data, Urban Climate, 23, 36–52,, 2018. 

Kotthaus, S. and Grimmond, C. S. B.: Identification of Micro-scale Anthropogenic CO2, heat and moisture sources – Processing eddy covariance fluxes for a dense urban environment, Atmos. Environ., 57, 301–316,, 2012. 

Kotthaus, S. and Grimmond, C. S. B.: Energy exchange in a dense urban environment – Part I: Temporal variability of long-term observations in central London, Urban Climate, 10, Part 2, 261–280,, 2014a. 

Kotthaus, S. and Grimmond, C. S. B.: Energy exchange in a dense urban environment – Part II: Impact of spatial heterogeneity of the surface, Urban Climate, 10, Part 2, 281–307,, 2014b. 

Lipson, M.: Urban-PLUMBER: A multi-site model evaluation project for urban areas – Project Home,, last access: 1 July 2021. 

Lipson, M.: Code to process “Harmonized gap-filled datasets from 20 urban flux tower sites” for the Urban-PLUMBER projects, Zenodo [code],, 2022a. 

Lipson, M.: Code to produce figures in the manuscript: “Harmonized, gap-filled dataset from 20 urban flux tower sites”, Zenodo [code],, 2022b. 

Lipson, M., Grimmond, S., and Best, M.: A new multi-site evaluation project for modelling in urban areas, Urban Climate News, 15–16, (last access: 14 November 2022), 2020a. 

Lipson, M. J., Grimmond, S., Best, M. J., Abramowitz, G., Pitman, A. J., and Ward, H. C.: Urban-PLUMBER: A new evaluation and benchmarking project for land surface models in urban areas, EGU General Assembly 2020, Online, 4–8 May 2020, EGU2020-20987,, 2020b. 

Lipson, M., Grimmond, S., Best, M. J., Abramowitz, G., and Pitman, A. J.: Urban-PLUMBER—Evaluation and Benchmarking of Land Surface Models in Urban Areas, 100th American Meteorological Society Annual Meeting, Boston, USA, 2020, (last access: 14 November 2022), 2020c. 

Lipson, M., Grimmond, S., and Best, M.: Protocol for observational data used in Urban-PLUMBER, Zenodo,, 2021. 

Lipson, M., Grimmond, S., Best, M., Chow, W., Christen, A., Chrysoulakis, N., Coutts, A., Crawford, B., Earl, S., Evans, J., Fortuniak, K., Heusinkveld, B. G., Hong, J.-W., Hong, J., Järvi, L., Jo, S., Kim, Y.-H., Kotthaus, S., Lee, K., Masson, V., McFadden, J. P., Michels, O., Pawlak, W., Roth, M., Sugawara, H., Tapper, N., Velasco, E., and Ward, H. C.: Data for “Harmonized gap-filled dataset from 20 urban flux tower sites” for the Urban-PLUMBER project, Zenodo [data set],, 2022. 

Macdonald, R. W., Griffiths, R. F., and Hall, D. J.: An improved method for the estimation of surface roughness of obstacle arrays, Atmos. Environ., 32, 1857–1864,, 1998. 

Martens, B., Schumacher, D. L., Wouters, H., Muñoz-Sabater, J., Verhoest, N. E. C., and Miralles, D. G.: Evaluating the land-surface energy partitioning in ERA5, Geosci. Model Dev., 13, 4159–4181,, 2020. 

Masson, V., Gomes, L., Pigeon, G., Liousse, C., Pont, V., Lagouarde, J.-P., Voogt, J., Salmond, J., Oke, T. R., Hidalgo, J., Legain, D., Garrouste, O., Lac, C., Connan, O., Briottet, X., Lachérade, S., and Tulet, P.: The Canopy and Aerosol Particles Interactions in TOulouse Urban Layer (CAPITOUL) experiment, Meteorol. Atmos. Phys., 102, 135–157,, 2008. 

Masson, V., Heldens, W., Bocher, E., Bonhomme, M., Bucher, B., Burmeister, C., de Munck, C., Esch, T., Hidalgo, J., Kanani-Sühring, F., Kwok, Y.-T., Lemonsu, A., Lévy, J.-P., Maronga, B., Pavlik, D., Petit, G., See, L., Schoetter, R., Tornay, N., Votsis, A., and Zeidler, J.: City-descriptive input data for urban climate models: Model requirements, data sources and challenges, Urban Climate, 31, 100536,, 2020. 

McNorton, J. R., Arduini, G., Bousserez, N., Agustí-Panareda, A., Balsamo, G., Boussetta, S., Choulga, M., Hadade, I., and Hogan, R. J.: An Urban Scheme for the ECMWF Integrated Forecasting System: Single-Column and Global Offline Application, J. Adv. Model. Earth Sy., 13, e2020MS002375,, 2021. 

Menard, C. B., Essery, R., Krinner, G., Arduini, G., Bartlett, P., Boone, A., Brutel-Vuilmet, C., Burke, E., Cuntz, M., Dai, Y., Decharme, B., Dutra, E., Fang, X., Fierz, C., Gusev, Y., Hagemann, S., Haverd, V., Kim, H., Lafaysse, M., Marke, T., Nasonova, O., Nitta, T., Niwano, M., Pomeroy, J., Schädler, G., Semenov, V. A., Smirnova, T., Strasser, U., Swenson, S., Turkov, D., Wever, N., and Yuan, H.: Scientific and Human Errors in a Snow Model Intercomparison, B. Am. Meteorol. Soc., 102, E61–E79,, 2021. 

Menne, M. J., Durre, I., Vose, R. S., Gleason, B. E., and Houston, T. G.: An Overview of the Global Historical Climatology Network-Daily Database, J. Atmos. Ocean. Tech., 29, 897–910,, 2012. 

Menzer, O. and McFadden, J. P.: Statistical partitioning of a three-year time series of direct urban net CO2 flux measurements into biogenic and anthropogenic components, Atmos. Environ., 170, 319–333,, 2017. 

New, M., Hulme, M., and Jones, P.: Representing Twentieth-Century Space–Time Climate Variability. Part I: Development of a 1961–90 Mean Monthly Terrestrial Climatology, J. Climate, 12, 829–856,<0829:RTCSTC>2.0.CO;2, 1999. 

Nogueira, M.: Inter-comparison of ERA-5, ERA-interim and GPCP rainfall over the last 40 years: Process-based analysis of systematic and random differences, J. Hydrol., 583, 124632,, 2020. 

Nordbo, A., Järvi, L., Haapanala, S., Moilanen, J., and Vesala, T.: Intra-City Variation in Urban Morphology and Turbulence Structure in Helsinki, Finland, Bound.-Lay. Meteorol., 146, 469–496,, 2013. 

Novick, K. A., Biederman, J. A., Desai, A. R., Litvak, M. E., Moore, D. J. P., Scott, R. L., and Torn, M. S.: The AmeriFlux network: A coalition of the willing, Agr. Forest Meteorol., 249, 444–456,, 2018. 

Offerle, B., Grimmond, C. S. B., and Oke, T. R.: Parameterization of Net All-Wave Radiation for Urban Areas, J. Appl. Meteor., 42, 1157–1173,<1157:PONARF>2.0.CO;2, 2003. 

Offerle, B., Jonsson, P., Eliasson, I., and Grimmond, C. S. B.: Urban Modification of the Surface Energy Balance in the West African Sahel: Ouagadougou, Burkina Faso, J. Climate, 18, 3983–3995, 2005. 

Oke, T. R.: The urban energy balance, Prog. Phys. Geog., 12, 471–508,, 1988. 

Oke, T. R., Mills, G., Christen, A., and Voogt, J. A.: Urban Climates, 1st edn., Cambridge University Press, ISBN 978-1-107-42953-6,, 2017. 

Pastorello, G., Trotta, C., Canfora, E., Chu, H., Christianson, D., Cheah, Y.-W., Poindexter, C., Chen, J., Elbashandy, A., Humphrey, M., Isaac, P., Polidori, D., Reichstein, M., Ribeca, A., van Ingen, C., Vuichard, N., Zhang, L., Amiro, B., Ammann, C., Arain, M. A., Ardö, J., Arkebauer, T., Arndt, S. K., Arriga, N., Aubinet, M., Aurela, M., Baldocchi, D., Barr, A., Beamesderfer, E., Marchesini, L. B., Bergeron, O., Beringer, J., Bernhofer, C., Berveiller, D., Billesbach, D., Black, T. A., Blanken, P. D., Bohrer, G., Boike, J., Bolstad, P. V., Bonal, D., Bonnefond, J.-M., Bowling, D. R., Bracho, R., Brodeur, J., Brümmer, C., Buchmann, N., Burban, B., Burns, S. P., Buysse, P., Cale, P., Cavagna, M., Cellier, P., Chen, S., Chini, I., Christensen, T. R., Cleverly, J., Collalti, A., Consalvo, C., Cook, B. D., Cook, D., Coursolle, C., Cremonese, E., Curtis, P. S., D'Andrea, E., da Rocha, H., Dai, X., Davis, K. J., Cinti, B. D., Grandcourt, A. de, Ligne, A. D., De Oliveira, R. C., Delpierre, N., Desai, A. R., Di Bella, C. M., Tommasi, P. di, Dolman, H., Domingo, F., Dong, G., Dore, S., Duce, P., Dufrêne, E., Dunn, A., Dušek, J., Eamus, D., Eichelmann, U., ElKhidir, H. A. M., Eugster, W., Ewenz, C. M., Ewers, B., Famulari, D., Fares, S., Feigenwinter, I., Feitz, A., Fensholt, R., Filippa, G., Fischer, M., Frank, J., Galvagno, M., et al.: The FLUXNET2015 dataset and the ONEFlux processing pipeline for eddy covariance data, Sci. Data, 7, 225,, 2020. 

Pawlak, W., Fortuniak, K., and Siedlecki, M.: Carbon dioxide flux in the centre of Łódź, Poland – analysis of a 2-year eddy covariance measurement data set, Int. J. Climatol., 31, 232–243,, 2011. 

Peters, E. B., Hiller, R. V., and McFadden, J. P.: Seasonal contributions of vegetation types to suburban evapotranspiration, J. Geophys. Res.-Biogeo., 116, G01003,, 2011. 

Porson, A., Clark, P. A., Harman, I. N., Best, M. J., and Belcher, S. E.: Implementation of a new urban energy budget scheme in the MetUM. Part I: Description and idealized simulations, Q. J. Roy. Meteor. Soc., 136, 1514–1529,, 2010. 

Rew, R., Davis, G., Emmerson, S., Cormack, C., Caron, J., Pincus, R., Hartnett, E., Heimbigner, D., Appel, L., and Fisher, W.: Unidata NetCDF, UCAR [code],, 1989. 

Roth, M., Jansson, C., and Velasco, E.: Multi-year energy balance and carbon dioxide fluxes over a residential neighbourhood in a tropical city, Int. J. Climatol., 37, 2679–2698,, 2017. 

Schmid, H. P., Cleugh, H. A., Grimmond, C. S. B., and Oke, T. R.: Spatial variability of energy fluxes in suburban terrain, Bound.-Lay. Meteorol., 54, 249–276,, 1991. 

Schmid, H. P., Grimmond, C. S. B., Cropley, F., Offerle, B., and Su, H.-B.: Measurements of CO2 and energy fluxes over a mixed hardwood forest in the mid-western United States, Agr. Forest Meteorol., 103, 357–374,, 2000. 

Shi, Y., Zhang, Y., and Li, R.: Local-Scale Urban Energy Balance Observation under Various Sky Conditions in a Humid Subtropical Region, J. Appl. Meteor. Climatol., 58, 1573–1591,, 2019. 

Stagakis, S., Chrysoulakis, N., Spyridakis, N., Feigenwinter, C., and Vogt, R.: Eddy Covariance measurements and source partitioning of CO2 emissions in an urban environment: Application for Heraklion, Greece, Atmos. Environ., 201, 278–292,, 2019. 

Steeneveld, G.-J., Horst, S. van der, and Heusinkveld, B.: Observing the surface radiation and energy balance, carbon dioxide and methane fluxes over the city centre of Amsterdam, EGU General Assembly 2020, Online, 4–8 May 2020, EGU2020-1547,, 2020. 

Tang, Y., Sun, T., Luo, Z., Omidvar, H., Theeuwes, N., Xie, X., Xiong, J., Yao, R., and Grimmond, S.: Urban meteorological forcing data for building energy simulations, Build. Environ., 204, 108088,, 2021. 

Ukkola, A. M., Haughton, N., De Kauwe, M. G., Abramowitz, G., and Pitman, A. J.: FluxnetLSM R package (v1.0): a community tool for processing FLUXNET data for use in land surface modelling, Geosci. Model Dev., 10, 3379–3390,, 2017. 

Ukkola, A. M., Abramowitz, G., and De Kauwe, M. G.: A flux tower dataset tailored for land model evaluation, Earth Syst. Sci. Data, 14, 449–461,, 2022. 

United Nations, Department of Economic and Social Affairs, Population Division (UN): World Urbanization Prospects: The 2018 Revision (ST/ESA/SER.A/420), United Nations, New York, ISBN 978-92-1-148319-2, 2019. 

Valentini, R.: EUROFLUX: An Integrated Network for Studying the Long-Term Responses of Biospheric Exchanges of Carbon, Water, and Energy of European Forests, in: Fluxes of Carbon, Water and Energy of European Forests, edited by: Valentini, R., volume 163, Springer, Berlin, Heidelberg, 1–8, ISBN 978-3-662-05171-9,, 2003. 

Varquez, A. C. G., Kiyomoto, S., Khanh, D. N., and Kanda, M.: Global 1-km present and future hourly anthropogenic heat flux, Scientific Data, 8, 64,, 2021. 

Velasco, E. and Roth, M.: Cities as Net Sources of CO2: Review of Atmospheric CO2 Exchange in Urban Environments Measured by Eddy Covariance Technique, Geography Compass, 4, 1238–1259,, 2010. 

Velasco, E., Pressley, S., Grivicke, R., Allwine, E., Molina, L. T., and Lamb, B.: Energy balance in urban Mexico City: observation and parameterization during the MILAGRO/MCMA-2006 field campaign, Theor. Appl. Climatol., 103, 501–517,, 2011. 

Velasco, E., Perrusquia, R., Jiménez, E., Hernández, F., Camacho, P., Rodríguez, S., Retama, A., and Molina, L. T.: Sources and sinks of carbon dioxide in a neighborhood of Mexico City, Atmos. Environ., 97, 226–238,, 2014. 

Vickers, D. and Mahrt, L.: Quality Control and Flux Sampling Problems for Tower and Aircraft Data, J. Atmos. Ocean. Tech., 14, 512–526,<0512:QCAFSP>2.0.CO;2, 1997. 

Vitale, D., Fratini, G., Bilancia, M., Nicolini, G., Sabbatini, S., and Papale, D.: A robust data cleaning procedure for eddy covariance flux measurements, Biogeosciences, 17, 1367–1391,, 2020. 

Vuichard, N. and Papale, D.: Filling the gaps in meteorological continuous data measured at FLUXNET sites with ERA-Interim reanalysis, Earth Syst. Sci. Data, 7, 157–171,, 2015.  

Ward, H. C., Evans, J. G., and Grimmond, C. S. B.: Multi-season eddy covariance observations of energy, water and carbon fluxes over a suburban area in Swindon, UK, Atmos. Chem. Phys., 13, 4645–4666,, 2013. 

Weedon, G. P., Gomes, S., Viterbo, P., Shuttleworth, W. J., Blyth, E., Österle, H., Adam, J. C., Bellouin, N., Boucher, O., and Best, M.: Creation of the WATCH Forcing Data and Its Use to Assess Global and Regional Reference Crop Evaporation over Land during the Twentieth Century, J. Hydrometeorol., 12, 823–848,, 2011. 

World Meteorological Organization: Guide to meteorological instruments and methods of observation, World Meteorological Organization, Geneva, Switzerland, ISBN 978-92-63-10008-5, 2008. 

Yamamoto, S., Saigusa, N., Gamo, M., Fujinuma, Y., Inoue, G., and Hirano, T.: Findings through the AsiaFlux network and a view toward the future, J. Geogr. Sci., 15, 142–148,, 2005. 

Short summary
We describe a new openly accessible collection of atmospheric observations from 20 cities around the world, capturing 50 site years. The observations capture local meteorology (temperature, humidity, wind, etc.) and the energy fluxes between the land and atmosphere (e.g. radiation and sensible and latent heat fluxes). These observations can be used to improve our understanding of urban climate processes and to test the accuracy of urban climate models.
Final-revised paper