Clouds are often clustered. Examples of larger clusters are squall lines and the intertropical convergence zone. But also much smaller clouds like shallow trade wind cumuli are often seen clustered on a scale of several hundred kilometers. The understanding of these mesoscale patterns of shallow convection is still sparse. However, the ubiquity of these clouds and their reoccurring structure suggest that they play an important role in determining the radiative effects of the trade wind regimes . The EUREC4A (Elucidating the role of clouds–circulation coupling in climate) field campaign addresses among others, the question of which processes are at play that change the mesoscale appearance of shallow convection. Prior to the campaign, studies concentrated on the classification of mesoscale patterns based on satellite images by visual inspection , rule-based algorithms , supervised neural networks and unsupervised neural networks . Most of these methods focus on four specific mesoscale cloud patterns that have been identified as commonly occurring in the North Atlantic downwind trades . An overview of these four patterns which are named by their visual impressions as sugar, gravel, flowers and fish is given in Fig. .

These efforts show that the categorization of the mesoscale patterns is an elementary step in acquiring further knowledge about the cloud processes in the downstream trades; 50 scientists from 15 research institutes who were involved in the EUREC4A field campaign in January–February 2020 therefore participated in a joint online classification event. The results of this classification event are presented here. Due to the high attendance, the presented dataset can be interpreted as the common judgment of the EUREC4A science team on the mesoscale patterns of shallow convection in the wider trades during EUREC4A. The studied domain of 5–20∘ N, 62–40∘ W contains the entire research area, including both the Tradewind Alley in the north (12.5–14.5∘ N) and the Boulevard des Tourbillons close to the coast of South America . This dataset therefore allows for consistently communicating about the otherwise subjectively defined patterns and puts the measurements taken during EUREC4A into their mesoscale context to advance the process understanding of shallow convection in the trades. The need for a better process understanding is highlighted by a supplemental classification of storm-resolving simulations covering the same time period.

The paper is structured as follows: the dataset and its collection are described in Sect. . Potential use cases are described in Sect. . In Sect.  additional classification methods that are able to detect the four mesoscale patterns are applied to the EUREC4A time period and compared to the manual classifications described in this paper. We conclude with Sect. .

The manual classifications were gathered through the online platform https://www.zooniverse.org/ (last access: 1 July 2020), which has already been successfully used in an earlier project by . The platform makes it possible to crowdsource labels for, e.g., machine learning projects. Additional workflows can be defined to separate different image sources or to separate, for example, labels made during a practice run from those that belong to the actual classification. The former allowed everyone to familiarize themselves with the https://www.zooniverse.org/ platform without influencing the results.

For this dataset, we defined three workflows. Two workflows are based on satellite observations. The first one is based on the visible channels of both the GOES-16 (Geostationary Operational Environmental Satellite) ABI (Advanced Baseline Imager) and MODIS on board the Aqua and Terra satellites and is called EUREC4A VIS. The second one is based on the “clean” infrared channel (channel 13) of GOES-16 ABI only (EUREC4A IR). To test how well storm-resolving simulations are able to reproduce the observed patterns, we included another workflow, EUREC4A ICON, which is based on ICON (ICOsahedral Nonhydrostatic) storm-resolving simulations covering the EUREC4A time period. The simulation has a grid spacing of 1.25 km and was initialized daily by the European Centre for Medium-Range Weather Forecasts Integrated Forecasting System (ECMWF IFS) and forced hourly. Each run is 48 h long. The first 24 h were discarded to allow for spin-up.

To visualize the simulation output, we calculated a pseudo-albedo α by following the approximation of : 1τ=0.19⋅LWP56⋅N132α=τ6.8+τ, where τ is the optical depth, LWP is the liquid water vapor path and N is an assumed cloud droplet number density. Here a number density of 70 cm-3 is used, which is at the lower end of the observed concentrations .

All workflows are further described in Table .

On 24 March 2020 the international, virtual classification event was hosted with 50 scientists from 15 institutes participating in the creation of the pattern classifications. For a full day the participants classified patterns of shallow trade wind convection by labeling, i.e., drawing rectangles of variable sizes around, the four common types: sugar, gravel, flowers and fish .

In the end, over 12 500 labels were gathered. The observational workflows got the majority of the labels (see Fig. ) because the identification of the patterns in the model simulation was too demanding. The cloud features in the simulations had too little similarity with those found in nature. By comparing the distribution of labels in the individual workflows, Fig.  reveals that sugar was disproportionately rarely identified in the ICON simulations. The largest feature, like fish, however, has been identified more often. This supports the assumption that larger features are better reproduced in storm-resolving simulations than features of smaller scales, like sugar.

Because all users have familiarity with the patterns either by previous work and/or being involved in the classification event of , it can be assumed that the labels are of high quality. In addition, they were trained immediately before starting the classification through an online presentation to get familiar with the labeling interface on https://www.zooniverse.org/ and to once more refresh the different mesoscale cloud pattern categories. Compared to , where the focus has been to classify as many diverse cloud scenes as possible to capture nature's variability and thus serve as a better machine learning dataset, the aim for this dataset is to create a common classification dataset for the EUREC4A time period that participating scientists agree on and can directly be used in further studies. Therefore, the temporal frequency has been increased from daily cloud scenes to 2-hourly cloud scenes to reflect also the changes on the sub-daily scale such as those identified by . Due to this design difference, a single day is now on average classified 20 times in case of the visible workflow instead of just about 6 (3 per each daytime Aqua/Terra satellite overpass) as in . Each individual image is however still viewed about 3 times.

After the joint classification event, over 12 500 labels were processed to make them more user friendly, especially because the raw data miss temporal and geospatial information. The processing steps with the intermediate products are illustrated in Fig.  and described as follows:

Level 0. The level 0 dataset consists of the raw data output and originates from the Zooniverse platform. It consists of CSV (comma-separated values) files that contain entries for each workflow, image (subject) and classification including technical details like the time spent on drawing a specific label. Labels are given by their origin (x,y) and their height (h) and width (w) given in pixel coordinates.

The EUREC4A field campaign has been an international study with a wide range of research platforms and many minor objectives . This dataset covers not only the core area of the experiment but also the wider area and time period. While the participating research airplanes and drones were mostly staying in the trade winds, some research vessels conducted measurements as far south as 6.5∘ N.

This dataset gives the opportunity to study all these measurements in the context of the mesoscale patterns observed in the downwind trades. Due to the high subjectivity of these mesoscale cloud pattern definitions, it is of particular importance to discuss results based on a common consensus to keep studies comparable. The C3ONTEXT dataset can serve as such a reference for the period of the EUREC4A field campaign.

Figure  shows the mesoscale context for three platforms participating in EUREC4A and representing different measurement strategies (see , for a complete list of platforms). Based on the classifications of the infrared workflow, the daily changes in the cloud patterns become apparent. The infrared workflow is preferred over the visible one because it covers both day and night and is therefore able to also capture the dial nature of the patterns . Before the intense observation period (IOP), 20 January to 20 February, the prevailing cloud pattern at the Barbados Cloud Observatory was gravel. With the start of the IOP, fish was detected most in the research area. After 24 January, patterns were much less widespread and well defined such that there was more disagreement on the patterns. This transition is independent of the location. Both the platforms in the north as well as R/V Atalante, which sailed further south in the Boulevard des Tourbillons (eddy boulevard), which is defined by as the area of the coastline of Brazil where the North Brazil Current (NBC) rings occur, were embedded in a similar transition. The reduced frequency of classifications towards the end of the IOP around 15 February are caused by mid-level clouds obscuring the view of shallow convection.

In Appendix  the level 3 products are visualized for each studied day and can be used as a lookup table to quickly identify if a pattern has been identified at a specific time and place.

Several methods have been developed to describe the mesoscale structure of shallow convection (). Here we focus on the comparison of the manual classifications with two other methods that specifically aim to detect the four mesoscale patterns of the downwind trades as defined by and shown in Fig. . combined a measure of organization (Iorg, ) with the mean cluster size (S), while developed a deep neural network to detect the patterns. Although show that different combinations of metrics, like cloud fraction and fractal dimension, better describe the variance in a cloud field, the pair of Iorg and mean cluster size has been widely used and is considered here for better comparison.

To assess the agreement between the different classification methods, we compare the method of and a deep neural network based on that detects the patterns in geostationary infrared images of GOES-16 ABI instead of visible images taken on board the polar-orbiting satellites Terra and Aqua. It should be noted that this deep neural network has not been trained with the manual classifications presented here but with the manual classifications of which cover older years and included different geographical regions. The network is identical to the one used in .

Because the Iorg/S measure is sensitive to the domain size, we compute these metrics over a 10×10∘ sub-domain and consider only classifications within this domain for the comparison. Specifically, we focus on the region 10–20∘ N, 58–48∘ W. This domain size ensures that Iorg describes the organization on the mesoscale and the cloud patterns we are interested in. The method has been successfully used in this domain in and applied accordingly. Brightness temperatures between 280 and 290 K are regarded as clouds. Days where the 25th percentile of brightness temperatures within any satellite image is lower than 285 K are discarded to avoid a bias by high clouds.

Figure  shows the comparison of the different methods. From the appearance of the patterns, we expect gravel and flowers to be rather regularly distributed and therefore to have a lower Iorg compared to fish and sugar. It should be noted that Iorg is calculated based on a threshold in brightness temperature, and therefore only the deeper clouds in the sugar field are considered leading to a higher Iorg value than one would expect from an otherwise rather randomly distributed cloud field. The mean cluster size should be small for sugar and gravel and larger for flowers and fish.

Indeed, the pattern area fraction maximizes for each pattern in the respective quadrant of the Iorg/S space independent of the classification method. Gravel classifications dominate the lower-left quadrant; sugar dominates the upper-left quadrant; and fish dominates the upper-right one. Flowers are harder to associate with a quadrant, as they are more centered. In general, the Iorg/S distribution has a large range of values for small mean cluster sizes and narrows with increasing mean cluster size.

This is also in alignment with , where the lower-right quadrant includes not only flowers but also about 35 % of the fish cases (their Fig. 1c). Flowers' cloud entities are generally smaller than those of the fish pattern and therefore cannot separate very well from the other patterns.

Figure  reveals the time series of the area fraction covered by the mesoscale patterns within the region of 10–20∘ N, 58–48∘ W and identified by the different classification approaches. For a better comparison with the EUREC4A VIS workflow, Fig. 7 is only based on daytime classifications (12:00–20:00 UTC).

To convert the agreement on patterns in the level 3 dataset into actual classifications, we applied a threshold of 0.1 to the percent agreement so that at least 10 % of participants who viewed a given scene must have agreed on a pattern. Agreement below this threshold is treated as an unclassified region. This threshold should be seen as a lower bound and is only used to reduce the noise of the classifications. For comparison, we also apply a stricter threshold of 0.5, which removes most of the overlap (Fig. b). The overall day-to-day changes remain robust. Figure  shows that in January–February 2020 all patterns were dominant at least once. It also shows that abrupt day-to-day changes of dominant patterns are rather rare, and a smooth transition from gravel to fish to sugar to flowers is observed.

Overall, the different classification methods agree well with each other, and no large discrepancies are found. This confirms that these methods are valid for further analysis of mesoscale patterns. While the Iorg/S metric is computationally cheap and can be easily applied to different regions, it is less suitable for short time periods because the terciles that attribute the Iorg/S pairs to specific quadrants are not robust for a small sample size. Terciles from other studies can only be used if the resolution of the dataset is the same. For short time periods, like those of typical campaigns where people are interested in specific days or even sub-daily variations, manual classifications are advantageous, as they do not require a large sample size. The neural network approach presents a good opportunity to extend the classifications to different time periods and regions by using manual classifications as a training set. Here, the manual classifications served as an additional independent validation dataset and proved once more the capabilities of the neural network which has been trained on the dataset of .

The C3ONTEXT dataset including raw data is openly available on the Zenodo database (; 10.5281/zenodo.5979718, ). The source code necessary to generate the dataset is available at 10.5281/zenodo.5989155 together with examples on how to process the data and retrieve the classifications for any platform as shown in, e.g., Fig. .

Mesoscale patterns of shallow convection are a main focus of the EUREC4A field campaign that took place in January and February 2020. To gain a better process understanding of shallow convection in the trades, the classifications of mesoscale patterns offer the opportunity to study the measurements in their mesoscale context and thereby split the observations into less complex pieces and disentangle otherwise superimposed processes. Here, we present C3ONTEXT, a dataset on the common consensus on the mesoscale patterns which occurred during the broader EUREC4A time period and emerges from the manual classifications done by members of the EUREC4A science team.

C3ONTEXT reveals that all mesoscale cloud patterns are observed during the studied period of January and February 2020. However, in the intense observation period of the EUREC4A field campaign, 20 January to 20 February, gravel was only sporadically identified and not prevalent in the study area. In contrast, a week before the intense observation period, gravel dominated in the research area. Instruments that were already running at this time, like those at the Barbados Cloud Observatory, were able to gain measurements under the mesoscale influence of gravel and complement the measurements from the IOP.

The difficulties of the participants to classify the patterns in the output of a storm-resolving simulation demands further investigation into how well simulations can capture the variability of mesoscale patterns of shallow convection in the trades.

A comparison of the manual-classification approach with other methods used in the literature to identify the four mesoscale patterns of shallow convection reveals a generally good agreement and confirms the validity of the different approaches. Nevertheless, the manual classifications are beneficial for limited temporal and spatial studies especially when the classifications are done by a group of several trained scientists. This presents a way to gain a consensus of subjectively defined cloud patterns without an additional layer of complexity from a neural network or any other algorithm.

In general, it has been shown that with little effort, classifications of the cloud field are possible and can be a huge benefit for the community, encouraging this approach for future studies.