Fluxes at half-hourly resolution are currently only achieved by eddy covariance instruments that provide local measurements and spatial extensions are so far only possible by deployment of those instruments on globally distributed towers. Based on these in situ observations, we aim at predicting half-hourly fluxes globally and therefore also rely on the data obtained by the eddy covariance method at different sites. The eddy covariance method has revolutionized the study of land–atmosphere interactions by offering a means of continuously observing net land–atmosphere fluxes of CO2, latent heat, and sensible heat . By now, the flux towers are running for sufficient time to enable studies about the interannual variability in land-surface dynamics, but the temporal representativeness remains highly uneven . In our studies, we rely on data from 222 FLUXNET eddy covariance towers (see Appendix  for a full list of involved sites). All towers are typically equipped with a suite of comparable micrometeorological devices; i.e., instrumentation and data outputs are similar enough, such that training of machine learning methods on data from multiple different sites is possible. Gross carbon fluxes can be derived using different flux partitioning methods as described, for example, by or , and here we rely on the former method. In all our experiments, we only make use of measured fluxes; i.e., no gap-filling has been applied and gaps in the half-hourly flux data have simply been ignored (more information on the distribution of data gaps can be found in Appendix ).

As predictor variables, we use the ones selected by Table 2 in the RS+METEO setup that they use for estimating fluxes at daily resolution. For convenience, we have reproduced this table and put it in Appendix . Besides the plant functional type (PFT), the variables contain remote sensing data from MODIS satellites and meteorological data either in situ measured at the flux tower locations or from long-term time series of the ERA-Interim data set at daily resolution. It should be noted that only the mean seasonal cycles (and derived properties like amplitude, minimum, mean, and maximum) are taken into account for the vegetation indices (normalized difference vegetation index – NDVI, enhanced vegetation index – EVI) as well as for the normalized difference water index (NDWI), the fraction of absorbed photosynthetically active radiation (fAPAR), and the land surface temperature (LST). In contrast, the actual values of air temperature, global radiation, potential radiation, relative humidity, and of different water availability indices have been used. For detailed descriptions, we refer to the corresponding sections in the paper of Sect. 2.1.3 and 2.1.4.

In order to compute the global flux products at half-hourly resolution via upscaling, we require the predictor variables mentioned in the previous section at global scale, i.e., the variables of the RS+METEO setup from Table  in Appendix , which has been reproduced from Table 2 of . Concerning the remote sensing variables, MODIS observations are used to compute mean values for each PFT and each day aggregated to 0.5∘ spatial resolution. The distributions of each PFT stem from the MODIS collection 5 global land cover product of . Climatic data for the meteorological variables have been obtained from CRUNCEPv6

Data from CRUNCEPv6 have been obtained via personal correspondence with Nicolas Viovy (email: nicolas.viovy@lsce.ipsl.fr).

Given a training set X=x(i)∈IRD:i=1,2,…,N of N samples with each sample x being a vector consisting of D predictor variables x1,x2,…,xD and a corresponding real-valued target variable y∈IR with observations y1,y2,…,yN∈IR for the N training samples, the goal is to find a set of rules that allow for predicting y based on x. In the case of a decision tree, these rules are binary tests for individual predictor variables with simple thresholds. A hierarchical tree structure is built as shown in Fig.  by selecting at each node i a predictor variable di∈1,2,…,D and a threshold ti∈IR. The estimate of a node i is the average value y‾i of the observations computed from training samples that reach this node. The first node of a decision tree, called root node, contains all training samples, and hence the overall mean value y‾1=1N∑n=1Nyn of observations yn from all N training samples is an extremely coarse approximation that needs to be refined depending on the constellation of the input variables x.

Starting at node 1 in Fig. , the set of training samples is partitioned into two subsets, represented by nodes 2 and 3, based on the result of the binary test xd1≤t1. Both nodes, node 2 and node 3, have associated predicted outputs y‾2 and y‾3 that are computed as the average observation of samples that reach the corresponding node. The split parameters d1 and t1 are optimized such that the mean squared error for the training samples is minimized given the respective predictions from node 2 or node 3. Such splits are then computed for nodes 2 and 3 as well as for further derived nodes until a stopping criterion is fulfilled. Typical stopping criteria are (i) a split would create nodes with less than Nmin samples, (ii) the variance of the observations from samples in a node is smaller than some threshold σmin2, or (iii) a maximum depth dmax of the tree is reached. The depth of a tree is defined as the largest distance of a node to the root of the tree. Values of the parameters Nmin, σmin2, and dmax can be changed to obtain either smaller or larger trees, which allows for controlling the runtime of the algorithm and the trade-off concerning generalization and overfitting.

It is usually the case that multiple stopping criteria are tested and if one of them is fulfilled, the current node is not split but becomes a leaf node that stores a final output prediction. Learning a decision tree therefore consists of computing split parameters until only leaf nodes remain that are not split any further (Fig. ). Hence, one distinguishes between split nodes as the inner nodes of a tree and leaf nodes, each of which contains the estimated output for any sample that reaches this node.

To reduce overfitting to the training set, the learning process is carried out in a stochastic manner by introducing several types of randomization. Whenever split parameters need to be identified, only a random subset of the D predictor variables is taken into account. Furthermore, only a fixed amount of randomly chosen thresholds is tested. Both randomization techniques also lead to reduced computation costs compared to exhaustive search. To predict the output y* of a test sample x*, it is passed through the tree according to the evaluation of the split functions at the inner nodes starting at the root node. This is done until a leaf node ℓ is reached, whose precomputed output y‾ℓ is assigned to x. However, more accurate predictions can be achieved by considering an ensemble of randomized decision trees.

In his work about random forests, makes use of a technique called bagging that he has introduced before . Bagging is an acronym for bootstrap aggregating and stands for aggregating predictions of individual models that have been learned based on different sample sets built from the original training data set. More precisely, individual sample sets are constructed by random sampling with replacement from the original training set, which is commonly referred to as bootstrapping. If the training set contains N samples, it is possible that each of the sampled sets either contains also N samples (which produces different sets with individual instances occurring several times due to random sampling with replacement) or only a fraction ν of the N samples. In both cases, the random subset selections introduced by bagging additionally prevent overfitting to the training set. For bagging, predictions from an ensemble of individual models are used and an ensemble of randomized decision trees is called random forest, randomized decision forest, or random decision forest (RDF). Each tree in the ensemble is learned separately and independent from the other trees. Due to the involved randomization techniques during learning of a single tree described before, different trees contain different binary tests and provide different estimates for a single input sample x. The individual predictions of each tree are then aggregated to obtain a final result, which is typically carried out by simple averaging as shown in Fig. .

However, the number of trees Ntree is a hyperparameter whose value needs to be chosen in advance but good assignments depend on various aspects. Since pointed out that bagging leads to predictions which are more stable compared to a single model, especially if the decision function of the single model is highly instable with respect to the training set, a larger number of trees is in favor of higher stability. On the other hand, more trees are causing higher computational costs during both learning and testing. In addition, a saturation effect for the prediction accuracy can typically be observed for an increasing number of trees. Hence, accuracies obtained by cross-validation for different numbers of trees can help to identify this saturation and a proper value for Ntree.

Recall from the beginning of Sect.  that the two main challenges for upscaling diurnal cycles to global scale are the huge amount of data which needs to be handled as well as the mismatch of temporal resolutions between predictor variables and the target fluxes. The first approach for predicting diurnal cycles has the advantage that it allows for using only predictors of daily temporal resolution. This is very important, because daily (average) values are often less noisy with respect to measurement noise and the availability of daily values is much higher compared to half-hourly values, especially when considering global products with values for every grid cell. Furthermore, variables derived from remote sensing are often limited to daily temporal resolution meaning that they are not more frequent than once per 24 h. Therefore, the first prediction approach involves learning an individual regression model for each half hour of the day and as indicated at the beginning of Sect. , RDF regression models are used for handling large-scale data. A schematic overview for a single day and a diurnal cycle of GPP is shown in Fig. .

Even if one uses only predictor variables of daily temporal resolution which can be treated as constant for the whole day, different values of the target flux for different half hours of the day can be estimated. The reason is that the 48 different RDF models are learned with different values for the target output variable y, although the same values for the predictor variables x are used. For example, an RDF model that is learned for a half hour during night only covers the rather small range of observations y that can be observed at this time, while the range of observations around noon is typically much larger, especially during the growing season. Hence, the 48 RDF models and their estimated outputs differ only because of different observations y that are provided during learning together with the same set of samples X. Of course, it is also possible to incorporate predictor variables at half-hourly temporal resolution, which would directly fit to the resolution of the target flux. Such predictor variables could further enhance the distinction of individual half hours of a day and could lead to more accurate estimations. However, they are optional and not required for this prediction approach as indicated in Fig. .

In contrast to the first prediction approach, the second approach only uses a single regression model that is able to estimate different values for different half hours of the same day. It is then necessary that the distinction between these half hours is somehow encoded in the predictor variables, which is not the case if only predictors of daily resolution are incorporated. Therefore, this approach requires at least one predictor variable at half-hourly temporal resolution. Fortunately, the potential radiation (Rpot) can be calculated globally at half-hourly resolution, because it only depends on the time as well as the solar angle that is defined given the spatial position via latitude and longitude. Thus, the second approach with a single model, as visualized in Fig. , is therefore also applicable for upscaling diurnal cycles to global scale. Again, we make use of an RDF regression model due to its large-scale capabilities.

In addition, besides the potential radiation, its first-order temporal derivative can also be incorporated as an additional half-hourly predictor. This allows for a distinction between morning and afternoon via the sign of the derivative as well as for the distinction between day and night. The latter is achieved because Rpot is zero for many consecutive hours during night leading also to a value of zero for its derivative in contrast to nonzero values of the derivative in the daytime. For all our computations, we have always included this derivative in the case when we also used Rpot. The nice property of this approach is that information about the physical relationships between the predictor variables and the fluxes can be shared among different half hours during learning of the single regression model, which is not the case for individual models as mentioned in the previous section. This second prediction approach therefore seems to be more plausible from a physical perspective, because the distinction between different half hours of the day is made based on the data, e.g., (potential) radiation, and not enforced by learning independent regression models for each half hour.

Although meteorological variables such as air temperature or vapor pressure deficit (VPD) as well as incoming radiation are also potential candidates for predictors that encode subdaily variations in the fluxes, they are currently only available with a half-hourly resolution at individual sites, e.g., also measured at eddy covariance towers. Due to the missing half-hourly meteorological data products at a global scale, it is not possible to use these information for the global upscaling. However, since we are interested in whether such data products could be beneficial for upscaling diurnal cycles, we use the corresponding site-level data in our cross-validation analysis to get further insights. Hence, meteorological variables measured at the eddy covariance towers of FLUXNET can still be used for validating the upscaling approaches and evaluations of cross-validation experiments are presented in the next section.

We start with a short overview of the experiments that have been conducted in order to clarify our ideas and motivations behind them. In Sect. , we compare the two different prediction approaches for upscaling diurnal cycles that have been introduced in Sect. . Furthermore, we focus on comparing different sets of predictor variables, e.g., the effect of meteorological variables at half-hourly resolution on the prediction performances. Evaluations of the prediction performance for monthly average diurnal cycles derived from the half-hourly values are shown in Sect. . These average diurnal cycles per month nicely summarize the fluxes over a longer time period (one month) by still keeping a half-hourly pattern that allows for monitoring subdaily variations. In addition, averaging diurnal cycles for a specific month removes noise in the individual half-hourly measurements and reduces the effects of day-to-day variability, e.g., caused by cloud coverage, which allows for comparing the main characteristics of the observations and the predictions for the selected month. The evaluations of monthly average diurnal cycles play an important role with regard to our provided data products, since we also prepare derived products that only contain these monthly average patterns. With two additional experiments presented in Sect. , we want to demonstrate that the quality of our achieved predictions is not inherently limited by the presented upscaling approaches but rather by missing site-specific information and latent driving forces that are not encoded in the set of predictor variables that has been used. This is not a specific problem of upscaling diurnal cycles of fluxes at half-hourly resolution but a general challenge for all upscaling approaches that deal with carbon and energy fluxes, also at coarser timescales.

In the following, we compare the results of our presented prediction approaches for half-hourly GPP depending on different sets of predictor variables, which have been obtained by using the leave-one-site-out strategy explained in the beginning of Sect. . As the core for all sets of predictors, we include those variables that have been used for upscaling daily mean GPP values by . In fact, we use exactly the same set of predictors that corresponds to the RS+METEO setup which has been defined by Table 2 and this table can also be found in Appendix . Given the explanations of the first prediction approach in Sect.  with individual regression models for each half hour, we can directly use these predictor variables for estimating half-hourly GPP values. However, we also added the potential incoming radiation Rpot at half-hourly resolution to encode subdaily variations in the predictors as well as its first temporal derivative to distinguish between morning and afternoon. Furthermore, we have tested a third set of predictors by additionally incorporating meteorological variables with half-hourly resolution measured at FLUXNET tower sites. The added variables are air temperature, vapor pressure deficit, and incoming global radiation. In a nutshell, the three sets of predictor variables consist of (i) daily predictors, (ii) daily predictors + half-hourly Rpot, and (iii) daily predictors + half-hourly Rpot + half-hourly meteorological predictors, besides static predictors like PFT that are used in all three cases. The second and third set are also used in the experiments for the second prediction approach that includes only a single regression model, because half-hourly information is encoded in some of the predictor variables.

In Fig. , we have visualized the results for all sites as well as for selected FLUXNET towers. Only for comparison to the leave-one-site-out experiments, we also included the modeling efficiency of a gap-filling algorithm as a potential upper bound for our predictions. In fact, each measured value is also estimated by a gap-filling algorithm that makes use of flux measurements at the same site under similar climate conditions and hence provides only a theoretical baseline, because it can not be applied for predicting fluxes at locations without any observations. First, we focus on the leftmost group of bars in Fig. , which shows the modeling efficiencies for all sites. Looking at the results for the first prediction approach with individual models for each half hour, including half-hourly Rpot only slightly improves the average performance (0.67 compared to 0.66), which is probably also caused by the stochastic nature of the RDF learning algorithm. However, including the meteorological predictors at subdaily temporal resolution leads to an increase in the performance to 0.70 modeling efficiency. A similar improvement can be observed for the second prediction approach with a single model for all half hours of the day, because the modeling efficiency increases from 0.67 to 0.71 when including half-hourly meteorological data. This highlights that varying subdaily meteorological conditions has a clear impact on predicting the diurnal cycles of GPP fluxes.

On the one side, half-hourly Rpot has almost no influence on the accuracy of the predictions but on the other side, it allows for applying the second prediction approach with only a single regression model. Hence, it may seem more natural from a physical perspective to distinguish individual half hours of the day by the provided half-hourly Rpot rather than enforcing the distinction by separately learned regression models. Comparing both prediction strategies, they achieve similar performances when using the same set of predictor variables. Since it is more convenient from a technical perspective to only handle a single regression model instead of 48 different models, the evaluations in the following sections will focus on the second prediction approach with a single RDF model that is suitable for predicting values at every half hour of the day. It is interesting to note that relative performance differences between the two prediction approaches and among the different sets of predictor variables look very similar when considering single sites only. In all our cross-validation experiments, the best prediction accuracies are always achieved by including half-hourly meteorological variables in the set of predictors. However, absolute performance values vary among sites. As shown in Fig. , the accuracies at the sites CA-Man and DE-Hai are between 0.80 and 0.90 modeling efficiency, whereas lower performances (between 0.60 and 0.80) have been achieved for predicting GPP at FR-Pue, IT-Cpz, and US-Goo. Moreover, the fluxes at US-Var seem to be very difficult to estimate, since only modeling efficiencies between 0.20 and 0.30 were obtained.

To further highlight the difference in the predictions when half-hourly meteorology is encoded in the driver variables, we visualize all half-hourly estimations over one year at a specific site using fingerprint plots. A fingerprint in this context is a plot with 365 rows corresponding to 365 days of a year and 48 columns corresponding to 48 half hours of each day such that one fingerprint contains all half-hourly values of a whole year and shows characteristic patterns for the selected site, e.g., length of the growing season. In Fig. , the estimations of half-hourly GPP with and without half-hourly meteorological predictors are shown in two individual fingerprint plots and their difference is indicated in a third plot. As expected, the predictions based on half-hourly meteorology contain much more short-term fluctuations during single days, whereas smoother estimations are obtained when only half-hourly Rpot is used as a subdaily driver. This can also be observed from the difference of the two fingerprints. Hence, half-hourly meteorological predictor variables are required to better capture high-frequency changes in the fluxes on subdaily timescales. In the following, we take a closer look at average diurnal cycles per month.

For visual inspection purposes, it is useful to look at average diurnal cycles for individual months at specific sites. Example plots are shown in Fig. . They show that our predictions are able to produce the typical shapes of diurnal courses which are in line with corresponding observations. For the depicted predictions, only observations from other sites have been used to learn the regression models. It is important to note that averaging diurnal cycles within a month reduces noise in the observations as well as in the predictions of a single day, but also smoothens high-frequency short-term fluctuations, e.g., due to (partial) cloud coverage, and yet decreases the influence of day-to-day variations. Hence, these mean diurnal courses are more stable and an evaluation of averaged predictions with respect to averaged observations for all sites leads to larger modeling efficiencies compared to those reported in the previous section. An overview of modeling efficiencies when comparing all half-hourly values versus only looking at the average diurnal cycles is given in Table .

In fact, modeling efficiencies for monthly average diurnal cycles increase on average across all sites to a range between 0.78 and 0.80 depending on the set of predictor variables with the best results being accomplished again by incorporating half-hourly meteorological data. This can also be observed from Fig. , which is organized in the same way as Fig.  but contains the achieved modeling efficiencies for comparing monthly average diurnal cycles of observations and predictions. For the monthly mean diurnal courses, the difference between only using half-hourly Rpot or also including half-hourly meteorology is not so large anymore compared to the evaluations for all half-hourly values. This holds for both the overall accuracies for all sites as well as for single selected sites. As previously mentioned, the reason is that averaging fluxes within a month reduces the effect of short-term fluctuations on subdaily timescales. Therefore, if one is only interested in monthly average diurnal cycles, the results obtained by using daily predictors and half-hourly Rpot are only slightly worse compared to including half-hourly meteorology and for some sites, the prediction performances are even on the same level of accuracy. This is important to know, since we also provide a derived product from our global half-hourly fluxes that contains the monthly average diurnal cycles globally at the same spatial resolution (Sect. ).

However, the average diurnal cycles can also be used to identify potential problems of the predictions. In Fig. , mean diurnal courses of several months at the sites FR-Pue and IT-Cpz are shown. It can be observed for both sites that the averaged observations are lower in the summer months compared to the corresponding predictions. In other words, the regression models overestimate GPP during these months. We believe that this is caused by the fact that our current prediction models are not able to cope with seasonal droughts, which is not a specific problem of diurnal upscaling but a challenge that every upscaling approach for carbon and energy fluxes needs to tackle. Although the observations show decreased productivity due to drought stress in summer, the regression models still estimate large amplitudes of the diurnal cycles, i.e., a larger productivity. One reason for this behavior could be the insufficient characterization of water availability that is present in the set of predictor variables. Currently, we plan to investigate this issue in further research. In the following, we show that our current sets of predictor variables are lacking some site-specific information, probably not only with respect to water storage capacities.

In order to gain any insights into whether site-specific information is currently not well represented in the predictors, we have conducted two auxiliary experiments. During the first experiment, we additionally estimate GPP fluxes at each site in a leave-one-month-out setup and compare the resulting predictions with those of the leave-one-site-out setup. For the leave-one-month-out estimations, we learn and test regression models for each month at each site separately. Furthermore, each regression model for each month is only learned with data from the same site but measured in different months (and years). Hence, the regression models are highly site-specific, since only correspondences between predictor variables and GPP fluxes at a single site are used and predictions are made at the same site but in a different time period. As a result, we have observed improved flux estimations, which is shown exemplarily in Fig.  for IT-Cpz. It can be clearly seen that the gaps between averaged observations and averaged predictions are getting smaller and mostly almost disappear; i.e., the predictions match the observations much better in the leave-one-month-out setup. In terms of modeling efficiency, the performances increase to a range between 0.75 and 0.79 when comparing all individual half-hourly predictions from the leave-one-month-out setup at all sites with the observations (best performance with leave-one-site-out is 0.70). Regarding the comparison of averaged predictions and averaged observations within each month as presented in the previous section, the leave-one-month-out setup leads to modeling efficiencies between 0.87 and 0.89. This is clearly larger than the results of the leave-one-site-out-experiment (best performance: 0.80). Table  allows for a direct comparison of the results from the leave-one-site-out and the leave-one-month-out experiments using both prediction approaches with the best set of predictor variables, i.e., daily predictor variables, half-hourly Rpot, and half-hourly meteorological variables.

This table also contains the prediction performances obtained from a second experiment, in which we have used the daily GPP as an additional daily predictor for our regression models in the leave-one-site-out setup. Of course, this is only possible in the cross-validation analysis where we actually have the daily averages of GPP, but the following evaluation reveals interesting insights. Using the daily average GPP basically incorporates information about the amplitudes of the diurnal cycles, hence drought effects of reduced productivity can directly be observed in this additional predictor variable. First of all, it can be seen in Fig.  that using the daily GPP as an additional predictor clearly improves the predictions at FR-Pue during summer months. Especially the decreased productivity in July and August 2005 can be nicely predicted by the regression models. Since the daily GPP as an additional predictor constrains the size of the peak in a diurnal cycle, the predictions become much more powerful and the characteristic shapes of the diurnal cycles can be produced. The modeling efficiencies are even larger than those obtained with the leave-one-month-out setup. They are in the range of 0.83 to 0.87 for all half-hourly values at all sites, which is comparable with the performance of the gap-filling algorithm that has been included as an additional reference in Fig.  as well as in Table . The gap-filling also achieves a 0.87 modeling efficiency; i.e., the upper performance limit shown as a green bar in the leftmost group in Fig.  can be obtained by including the daily GPP as an additional predictor. Regarding monthly averaged diurnal cycles, a modeling efficiency of up to 0.94 is obtained by the regression models that use daily GPP as an additional predictor, while gap-filling reaches 0.93. This is also summarized in Table .

From this experiment, we can conclude that the problems for predicting diurnal cycles of GPP are mainly caused by the lack of estimating the daily mean GPP properly. If the daily mean is given, predictions of half-hourly values are much more accurate. Hence, the main problems for the upscaling of half-hourly fluxes are not related to producing the right shapes of the diurnal courses, but turn out to be problems of estimating the correct amplitudes. These are then the same problems as for upscaling daily average values (or fluxes at coarser timescales) and are not introduced by the step of going to a larger temporal resolution in terms of half hours.

In this section, we want to shortly summarize the main findings from our cross-validation experiments. First, we have seen that it does not really matter which of the two proposed prediction approaches we are using, since prediction performances hardly differed between the single model approach and the individual model approach. We prefer to use the single model approach, because it seems to be more plausible from a physical perspective to make distinctions between half hours of a day by the information encoded in the predictor variables and half-hourly Rpot can always be used for this purpose. Second, including half-hourly meteorological information in the predictors clearly helps to improve the prediction performances for fluxes on the half-hourly timescale. However, for monthly average diurnal cycles the differences are not so prominent anymore and estimations based on half-hourly Rpot as the only predictor at half-hourly resolution may be sufficient for analyzing the monthly patterns. Third, we have shown that the main problem for upscaling half-hourly fluxes is not the fact that we increase the temporal resolution, since we are able to reproduce the characteristic subdaily patterns. Moreover, we are lacking additional information in the predictors that encode site-specific characteristics as well as certain special conditions like seasonal droughts. This currently prevents us from obtaining the correct day-to-day variability and also, in the end, the correct interannual variability. However, these are also problems that need to be tackled when an upscaling of carbon and energy fluxes at coarser timescales is considered . In the following section, we summarize the results from our cross-validation experiments for all the four fluxes (GPP, NEE, LE, H) with the setup that has been used to compute the global half-hourly data products.

While the previous sections validate the presented prediction approaches and point to potential problems in the estimation of half-hourly fluxes, we also decided to produce the first global products of half-hourly GPP and NEE, as well as LE and H that will be described in the next section. So far, the analyses have shown that best predictions are obtained by incorporating meteorological variables at half-hourly resolution, but such data products are not available at a global scale. Therefore, we have computed the global products only based on the daily predictors of the RS+METEO setup from Table 2, which can also be found in Appendix , as well as using half-hourly values of Rpot and its first temporal derivative. The used data sources have been described in Sect.  and the set of daily predictors varies between carbon and energy fluxes as indicated within the aforementioned table.

Furthermore, we have decided to use the second prediction approach (Sect. ) by learning one single regression model that is suitable for all half hours of the day. For us, it seems more natural from a physical perspective to distinguish between different half hours of a day by (potential) radiation as an additional variable rather than enforcing the distinction with individual models for each half hour as it is done in the first prediction approach (Sect. ).

In Table , we report the corresponding prediction performances from the leave-one-site-out cross-validation experiments for this setup, i.e., for the selected set of predictor variables and the single regression model approach. The modeling efficiencies for both individual half-hourly values and monthly average diurnal cycles are stated. Comparing these values, we observe that the accuracies for predicting energy fluxes are higher compared to those for the carbon fluxes. Half-hourly values of the sensible heat flux can be best estimated by achieving a modeling efficiency of 0.77 across all sites. On the other hand, net ecosystem exchange has only been predicted with a modeling efficiency of 0.61. This performance is lower compared to the one for gross primary production (0.67), probably due to missing information in the predictor variables for the respiration component of NEE. For all four fluxes, the modeling efficiencies are higher when comparing monthly average diurnal cycles of observations and predictions. The main reasons, as also mentioned in Sect. , are the reduction of noise and the smoothing of short-term fluctuations at subdaily timescales due to the averaging. In the following section, we present the global half-hourly flux products that have been calculated with the upscaling approach and the setup of this section.

Cutouts of the global products are visualized in Fig. , where we have selected 14 June 2014 at 13:00 UTC in the time domain. Global maps of GPP and NEE are shown in the top row of Fig.  and one can nicely distinguish daytime from nighttime for individual regions around the world. Furthermore, selected locations are highlighted and all half-hourly values of the year 2014 for these grid cells are summarized in fingerprint plots, which allow for identifying different characteristics at the corresponding places due to the different patterns in these plots. The fingerprints provide a nice overview of the half-hourly fluxes over a whole year and different lengths of the growing season as well as varying lengths of the day (time between sunrise and sunset) can directly be observed. Corresponding maps for LE and H with fingerprint plots for the same locations are shown in the bottom row of Fig. . Larger values of LE compared to H at this specific point in time can be observed in western, central, and eastern Europe as well as in the tropical regions of Africa, whereas it is vice versa on the Iberian Peninsula as well as in the northern and southern regions of Africa.

Besides the fingerprint plots summarizing a whole year of half-hourly values for a specific location, it is also possible to compute diurnal amplitudes for each grid cell from the global products. We again picked GPP acting as an example for all the fluxes and determined maximum diurnal amplitudes within each month. In Fig. , the maximum diurnal amplitudes of GPP are visualized for June and December 2014 with a logarithmic color scale. These months have been chosen to indicate differences between summer and winter. The biosphere at the Northern Hemisphere is quite active in June showing large amplitudes, whereas maximum amplitudes are close to zero at most of the grid cells of the Northern Hemisphere in December. In the tropics, amplitudes of GPP do not vary much between June and December with values around 30 µmolm-2s-1. As expected, the behavior in the Southern Hemisphere is opposite to the Northern Hemisphere; i.e., the productivity in the Southern Hemisphere is higher in December compared to June.

Furthermore, we have been interested in the maximum flux at each spatial position. These statistics have been calculated among all the years 2001 to 2014 to produce a single map of maximum half-hourly values for each flux. The results are shown in Fig. . Those maximum values denote the capabilities of each flux at each grid cell. For GPP, the hot spots with maximum capacities are in the corn belt of the USA, in eastern China, and in the tropical regions. Largest values of NEE are obtained in the tropics as well, especially in the Amazon. Regarding the energy fluxes, it is not so easy to identify single hot spot regions since large values of LE or H are widespread. However, distinct spatial patterns can be observed in all maps of the maximum fluxes.

Finally, we want to compare our global product of NEE with an ensemble of atmospheric CO2 inversions that contains CarbonTracker , CarbonTracker Europe , the Jena CarboScope inversion scheme (s99_V3.6) from , and MACC-II of that has also been used by . We consider temporal aggregations with a monthly resolution and have computed the mean seasonal cycle (MSC) from the years 2001 to 2010 for each grid cell. In the spatial domain, we have then performed an averaging step according to the 11 TransCom land regions : North American Boreal, North American Temperate, South American Tropical, South American Temperate, Northern Africa, Southern Africa, Eurasian Boreal, Eurasian Temperate, Tropical Asia, Australia, and Europe.

The upper panel in Fig.  contains the MSC of our NEE product for the 11 TransCom land regions as well as the average MSC from the ensemble of atmospheric inversions. For the latter, the shaded regions are spanned by minimum and maximum values per month. Since it is known that upscaling methods tend to overestimate the carbon uptake of the biosphere (too large carbon sink) compared to atmospheric inversions, we have also subtracted the mean value from all curves in the upper panel and show the differences from the corresponding mean values in the lower half in order to see whether the patterns in the MSC are matching between the upscaled product and the inversions. One can nicely see from the plots in the lower half that this is the case for most of the regions, except for the tropics where seasonality is also small. In the upper panel, we also observe the largest discrepancies between the upscaled NEE product and the results from the atmospheric inversions in the tropical regions. However, this is a known problem for the upscaling approaches that rely on flux tower measurements and not a specific problem of our product. The reason is not yet clarified in the community but one issue is related to the fact that there are only very few flux tower sites in tropical regions, which can also be observed from Fig. .

For some regions, such as Southern Africa and South American Temperate, mismatches between the inversions and the flux tower upscaling might also be due to contributions of fire emissions which are “seen” by the atmosphere but not in our approach. Overall, the patterns of the MSC in most of the regions are very similar to the results of the atmospheric inversions. This is remarkable given that the two approaches and data sources are entirely independent. Thus, our upscaling product has the potential to provide further constraints for the atmospheric inversion methods with the benefit of high resolution in both space and time.

We use the atmospheric inversions as an independent benchmark here, even though a number of uncertainties also apply to those. To put the agreement of the upscaling with the inversions into context of agreement among inversions, we display the values of pairwise correlation coefficients for the MSC of the different inversion methods together with the correlation coefficients between the MSC of each inversion approach and our upscaling product for each TransCom land region in Fig. . Overall, the agreement of our upscaling approach with the inversions is comparable with the agreement among different inversions suggesting promise for a synergistic joint usage of both approaches. There is very large consistency among inversions and with the upscaling approach for the regions North American Boreal, North American Temperate, Eurasian Boreal, and Europe. Interestingly, the upscaling approach is more consistent with the inversions than among the inversions for the Eurasian Temperate region. We observe that the seasonality of NEE in the tropical regions is not well constrained by each approach.