A compilation of global bio-optical in situ data for ocean colour satellite applications – version three

A global in situ data set for validation of ocean colour products from the ESA Ocean Colour Climate Change Initiative (OC-CCI) is presented. This version of the compilation, starting in 1997, now extends to 2021, which is important for the validation of the most recent satellite optical sensors such as Sentinel 3B OLCI and NOAA-20 VIIRS. The data set comprises in situ observations of the following variables: spectral remote-sensing reflectance, concentration of chlorophyll-a, spectral inherent optical properties, spectral diffuse attenuation coefficient, and total suspended matter. Data were obtained from multi-project archives acquired via open internet services or from individual projects acquired directly from data providers. Methodologies were implemented for homogenization, quality control, and merging of all data. Minimal changes were made on the original data, other than conversion to a standard format, elimination of some points, after quality control and averaging of observations that were close in time and space. The result is a merged table available in text format. Overall, the size of the data set grew with 148 432 rows, with each row representing a unique station in space and time (cf. 136 250 rows in previous version; Valente et al., 2019). Observations of remote-sensing reflectance increased to 68 641 (cf. 59 781 in previous version; Valente et al., 2019). There was also a near tenfold increase in chlorophyll data since 2016. Metadata of each in situ measurement (original source, cruise or experiment, principal investigator) are included in the final table. By making the metadata available, provenance is better documented and it is also possible to analyse each set of data separately. The compiled data are available at https://doi.org/10.1594/PANGAEA.941318 (Valente et al., 2022).

1 Introduction

Data collected by satellite ocean colour sensors provide synoptic observations on ocean productivity and the variability of marine environment at high spatial and temporal resolutions. Ocean colour data, recognized as Essential Climate Variables by the Global Climate Observation System, are invaluable to address key issues, such as the detection of marine ecosystem modifications due to climate change, the study of the global carbon cycle, and the assessment of coastal water quality degradations (IOCCG, 2008; McClain, 2009). A main goal of the ESA Ocean Colour Climate Change Initiative (OC-CCI) was to generate a suite of ocean colour products for use in climate studies (Sathyendranath et al., 2019). For this purpose, the existing major data streams for ocean colour were blended into a coherent ocean colour data record. Currently, data from five ocean colour sensors are being merged: the Sea-viewing Wide Field-of-view Sensor (SeaWiFS) of NASA, the Medium Resolution Imaging Spectrometer (MERIS) of ESA, the MODerate resolution Imaging Spectro-radiometer (MODIS) of NASA, the Visible Infrared Imaging Radiometer Suite (VIIRS) of NASA and NOAA, and the Ocean and Land Colour Instrument (OLCI) of ESA. For the validation of the ESA OC-CCI satellite products, a compilation of in situ bio-optical data was produced. This paper presents that compilation.

There are several sets of in situ bio-optical data worldwide suitable for validation of ocean colour satellite data. While some are managed by the data producers, others are in international repositories with contributions from multiple scientists. Many have rigid quality controls and are built specifically for ocean colour validation. The use of only any one of these data sets would limit the amount of data in validation exercises. It is therefore vital to merge all these in situ data sets to maximize the number of matchups available for validation, with wider distribution in time and space, and consequently to reduce uncertainties in the validation exercise. However, merging several data sets together can be a complicated task. First, it is necessary to acquire and harmonize all data sets into a single standard format. Second, during the merging, duplicates between data sets must be identified and removed. Third, the metadata should be propagated throughout the process and made available in the final merged data set. Ideally, the compiled merged data set would be made available as a simple text table to facilitate ease of access and manipulation. In this work, such unification of multiple data sets is presented. This was done for the validation of the ESA OC-CCI ocean colour products, but with the intent to also serve the broader user community.

A merged data set is not without drawbacks: it is likely to be large (with hundreds of thousands of observations) and so not always easy to manipulate; because the merging is done on pre-existing, processed databases, it is not possible to have full control of the whole processing chain. Hence, the data set would be a collection of observations collected by several investigators using different instruments, sampling methods, and protocols, which might eventually have been modified by the processing routines used by the repositories or archives. To minimize these potential drawbacks, we have, for the most part, incorporated only data sets that have emerged from the long-term efforts of the ocean colour and biological oceanographical communities to provide scientists with high-quality in situ data, and implemented additional quality checks on the data to enhance confidence in the quality of the merged product. Nevertheless, it is still recognized that different and unpredictable uncertainties may affect data from the diverse sources due to the use of a variety of field/laboratory instruments, methods, and data reduction schemes.

Methodologies used for data harmonization and integration as well as a description of the acquired individual data sets are provided in Sect. 2. Geographic distribution and other characteristics of the final merged data set are shown in Sect. 3, while Sect. 4 provides an overview of the data.

2 Data and methods

2.1 Pre-processing and merging

The compiled global in situ bio-optical data set described in this work has an emphasis, though not exclusively, on open-ocean data. It comprises the following variables: remote-sensing reflectance (“rrs”), chlorophyll-a concentration (“chla”), algal pigment absorption coefficient (“aph”), detrital and coloured dissolved organic matter absorption coefficient (“adg”), particle backscattering coefficient (“bbp”), diffuse attenuation coefficient for downward irradiance (“kd”) and total suspended matter (“tsm”). The variables “rrs”, “aph”, “adg”, “bbp”, and “kd” are spectrally dependent, and this dependence is hereafter implied. The data were compiled from 27 sources (MOBY, BOUSSOLE, AERONET-OC, SeaBASS, NOMAD, MERMAID, AMT, ICES, HOT, GeP&CO, AWI, ARCSSPP, BARENTSSEA, BATS, BIOCHEM, BODC, CALCOFI, CCELTER, CIMT, COASTCOLOUR, ESTOC, IMOS, MAREDAT, PALMER, SEADATANET, TPSS, and TARA), each one described in Sect. 2.2. The data sources in this work should also be viewed as groups of data that were acquired from a specific source, standardized with a specific method, and later merged into the compilation. The compiled in situ observations are essentially surface (i.e. no information depending on depth), have a global distribution, and cover the period 1997 to 2021. The listed variables, with the exception of total suspended matter, were chosen as they are the operational satellite ocean colour products of ESA OC-CCI project.

The compilation is provided in the format of three 2-dimensional main tables that relate to each other via one unique key identifying each row. The format of the tables is described in Appendix B. Despite being provided in three main tables, the compilation should still be viewed conceptually as one unique table, and as such it is still described in that way. The data set contains two flags: “flag_time” and “flag_chl_method”. The first is because three data sources were used (ESTOC, MAREDAT, and TPSS) where information on time (hour of the day) was not available. The time for these observations was set to 12:00:00 (UTC) and the observations were flagged with “1” in column “flag_time”. A second flag was necessary because in two data sources (ARCSSPP and SEADATANET) there was uncertainty on whether the compiled chlorophyll concentrations were measured using fluorometric, spectrophotometric, or HPLC (high-performance liquid chromatography) methods. The compiled chlorophyll observations from these two data sources were flagged with “1” in column “flag_chl_method” and were marked as “chla_fluor”.

This is the third version of the compilation. The first and second versions were described in Valente et al. (2016, 2019), respectively. Compared to the previous version (Valente et al., 2019), the present version contains more measurements of “rrs”, “chla”, and “aph”. The “rrs” stations increased by ∼15 % (i.e. from 59 781 to 68 641), resulting from updates of AERONET-OC, BOUSSOLE, MOBY, MERMAID, and AWI. The new stations are mainly for the period of 2019–2021 (previous version had “rrs” data until 2018). Regarding “chla”, a major increase in the number of recent observations was obtained. The previous version had “chla” data until 2017, with 533 stations for the period 2016–2017. The current version has 5140 stations for 2016–2021, which constitutes a near tenfold (964 %) increase since 2016. The new “chla” data originate from updates of BOUSSOLE, MERMAID, SeaBASS, HOT, AMT, PALMER, CCELTER, CALCOFI, AWI, and IMOS. As for the number of “aph” stations, it increased by ∼30 % (i.e. from 3293 to 4265), with most of the data between 2012–2020 (previous version finished in 2012). The new “aph” data come from updates of SeaBASS and AWI. Overall, the main objective of the present version was to populate the compilation with more recent data. Methodologies for data harmonization and integration (described below) have not been altered relative to the last version.

Remote-sensing reflectance is a primary ocean colour product defined as “rrs = Lw/Es”, where “Lw” is the upward water-leaving radiance and “Es” is the total downward irradiance at sea level. Another quantity that is often required is the “normalized” water-leaving radiance (“nLw”) (Gordon and Clark, 1981), which is related to remote-sensing reflectance via “rrs = nLw/Fo”, where “Fo” is the top-of-the-atmosphere solar irradiance. If not directly available, remote-sensing reflectance was calculated through the equations described above, depending on the format of the original data. The original data were acquired in an advanced form (e.g. time-averaged, extrapolated to surface) from nine data sources designed for ocean colour validation and applications (MOBY, BOUSSOLE, AERONET-OC, SeaBASS, NOMAD, MERMAID, COASTCOLOUR, TARA, AWI), therefore only requiring the conversion to a common format. In processing by space agencies, the quantity “rrs” is normalized to a single Sun-viewing geometry (Sun at zenith and nadir viewing) taking in account the bidirectional effects as described in Morel and Gentili (1996) and Morel et al. (2002). Thus, for consistency with satellite “rrs” product, the latter normalization was applied to the in situ “rrs”.

Chlorophyll-a concentration is a proxy measure for phytoplankton biomass and one of the most-widely used satellite ocean colour products (IOCCG, 2008). To validate satellite-derived chlorophyll-a concentration, two different variables were compiled: one of these represents chlorophyll-a measurements made through fluorometric or spectrophotometric methods, referred to hereafter as “chla_fluor”, and the other is the chlorophyll concentration derived from HPLC (high-performance liquid chromatography) measurements, referred to hereafter as “chla_hplc”. The chlorophyll data were compiled from the following 25 data sources: BOUSSOLE, SeaBASS, NOMAD, MERMAID, AMT, ICES, HOT, GeP&CO, AWI, ARCSSPP, BARENTSSEA, BATS, BIOCHEM, BODC, CALCOFI, CCELTER, CIMT, COASTCOLOUR, ESTOC, IMOS, MAREDAT, PALMER, SEADATANET, TPSS, and TARA. One requirement for “chla_fluor” measurements was that they were made using in vitro methods (i.e. based on extractions of chlorophyll-a). Although this severely decreased the number of observations, since in vivo fluorometry (e.g. fluorometers mounted on CTDs) is widely available in oceanographic databases, it was decided to exclude such data because of potential problems with the calibration of in situ fluorometer data. The variable “chla_hplc” was calculated by summing all reported chlorophyll-a derivatives, including divinyl chlorophyll-a, epimers, allomers, and chlorophyllide-a. The two chlorophyll variables are retained separately in the database to facilitate their use. HPLC measurements could be considered of higher quality, but fluorometric measurements are more numerous. Thus, one option for users is to use “chla_fluor” only when there are no “chla_hplc” measurements available. To be consistent with satellite-derived chlorophyll values, which are derived from the light emerging from the upper layer of the ocean, all chlorophyll observations in the top 10 m (replicates at the same depth, or measurements at multiple depths) were averaged if the coefficient of variation among observations was less than 50 %, otherwise they were discarded. The averages were then assigned to the surface. The depth of 10 m was chosen as a compromise between clear oligotrophic and turbid eutrophic waters. Other methods, such as chlorophyll depth-averages using local attenuation conditions (Morel and Maritorena, 2001), require observations at multiple depths, which, given our decision to use only in vitro measurements, would have considerably reduced the final number of observations.

Regarding the inherent optical properties (“aph”, “adg”, “bbp”), if not already calculated and provided in the contributed data sets, they were computed from related variables that were available: particle absorption (“ap”), detrital absorption (“ad”), coloured dissolved organic matter (CDOM) absorption (“ag”), and total backscattering (“bb”). The following equations were used: “adg = ad + ag”, “ap = aph + ad”, and “bb = bbp + bbw”. For the latter equation, the variable ”bbw” was computed using “bbw = bw/2”, where “bw” is the scattering coefficient of seawater derived from Zhang et al. (2009). The diffuse attenuation coefficient for downward irradiance (“kd”) did not require any conversion and was compiled as originally acquired. Observations of inherent optical properties (surface values) and diffuse attenuation coefficient for downward irradiance were acquired in total from six data sources designed for ocean colour validation and applications (SeaBASS, NOMAD, MERMAID, AWI, COASTCOLOUR, TPSS), thus already subject to the processing routines of these data sets. Concerning total suspended matter, these data were compiled as originally available from MERMAID and COASTCOLOUR.

The merged data set was compiled from 27 sets of in situ data, which were obtained individually either from archives that incorporate data from multiple contributors (SeaBASS, NOMAD, MERMAID, ICES, ARCSSPP, BIOCHEM, BODC, COASTCOLOUR, MAREDAT, SEADATANET), or from particular contributors, measurement programs, or projects (MOBY, BOUSSOLE, AERONET-OC, HOT, GeP&CO, AMT, AWI, BARENTSSEA, BATS, CALCOFI, CCELTER, CIMT, ESTOC, IMOS, PALMER, TPSS, TARA), and were subsequently homogenized and merged. Data contributors are listed in Table 2 and in the auxiliary material. There were methodological differences between data sets. Therefore, after acquisition, and prior to any merging, each set of data was pre-processed for quality control and converted to a common format. During this process, data were discarded if they had: (1) unrealistic or missing date and geographic coordinate fields; (2) poor quality (e.g. original flags) or method of observation that did not meet the criteria for the data set (e.g. in situ fluorescence for chlorophyll concentration); and (3) spuriously high or low data. For the last, the following limits were imposed: for “chla_fluor” and “chla_hplc” [0.001–100] mg m⁻³; for “rrs” [0-0.15] sr⁻¹; for “aph”, “adg”, and “bbp” [0.0001–10] m⁻¹; for “tsm” [0–1000] g m⁻³; and for “kd” [(aw(λ)-10] m⁻¹, where “aw” is the pure water absorption coefficients derived from Pope and Fry (1997). Also, during this stage, three metadata strings were attributed to each observation: “dataset”, “subdataset”, and “contributor”. The “dataset” contains the name of the original set of data and can only be one of the following: “aoc”, “boussole”, “mermaid”, “moby”, “nomad”, “seabass”, “hot”, “ices”, “amt”, “gepco”, “arcsspp”, “awi“, “barentssea“, “bats“, “biochem“, “bodc“, “calcofi“, “cc“, “ccelter“, “cimt“, “estoc“, “imos“, “maredat“, “palmer“, “seadatanet“, “tpss“, and “tara”. The “subdataset” starts with the “dataset” identifier and is followed by additional information about the data, as <dataset>_<cruise/station/site>) (e.g. “seabass_car81”). The “contributor” contains the name of the data contributor. An effort was made to homogenize the names of data contributors from the different sets of data. These three metadata are the link to trace each observation to its origin and were propagated throughout the processing. Finally, this processing stage ended with each set of data being scanned for replicate variable data and replicate station data, which when found, were averaged if the coefficient of variation was less than 50 %, otherwise they were discarded. Replicates were defined as multiple observations of the same variable, with the same date, time, latitude, longitude, and depth. Replicate station data were defined as multiple measurements of the same variable, with the same date, time, latitude, and longitude. For the latter case, a search window of 5 min in time and 200 m in distance was given to account for station drift. A small number of observations that were identified as replicates had a different “subdataset” identifiers (i.e. different cruise names). These observations were considered suspicious if the values were different and discarded. If the values were the same, one of the observations was retained. This possibly originated from the same group of data being contributed to an archive by two different data contributors.

Table 1 The standard variables, nomenclatures, and units in the final table.

Once a set of data was homogenized, its data were integrated into a unique table. This final merging focused on the removal of duplicates between the sets of data. Although some duplicates are known (e.g. MOBY, BOUSSOLE, AERONET-OC, and NOMAD data are found in SeaBASS and MERMAID), others are unknown (e.g. how many of GeP&CO, ICES, AMT, and HOT are within NOMAD, SeaBASS, and MERMAID). Therefore, duplicates were identified using the metadata (“dataset” and “subdataset”) when possible, and temporal–spatial matches, as an additional precaution. For temporal–spatial matches, several thresholds were used, but typically 5 min and 200 m were taken to be sufficient to identify most duplicated data, which reflected small differences in time, latitude, and longitude, between the different sets of data. Larger thresholds were used in some cases as a cautionary procedure. This was the case when searching for NOMAD data in other data sets, because NOMAD includes a few cases where merging of radiometric and pigment data was done with large spatial–temporal thresholds (Werdell and Bailey, 2005). A large temporal threshold was also used when integrating observations from the three data sources that did not have time available (ESTOC, MAREDAT, and TPSS). In regard to all data, if duplicates were found, data from the NOMAD data set were selected first, followed by data from individual projects or contributors (MOBY, BOUSSOLE, AERONET-OC, AMT, HOT,GeP&CO, AWI, BARENTSSEA, BATS, CALCOFI, CCELTER, CIMT, ESTOC, IMOS, PALMER, TPSS, and TARA) and finally for the remaining data sets (SeaBASS, MERMAID, ICES, ARCSSPP, BIOCHEM, BODC, COASTCOLOUR, MAREDAT, and SEADATANET). This procedure was chosen to preserve the NOMAD data set as a whole, since it is widely used in ocean colour validation. It should be noted that, by this procedure, data from individual projects or contributors may be listed under NOMAD (e.g. some PALMER data are found in NOMAD with metadata string “nomad_palmer_lter”). After giving priority to NOMAD, the priority was generally given to data from individual projects or contributors, but due to an incremental approach where only new data are added to previous versions of the compilation, some data from individual projects or contributors (BATS, CALCOFI, CIMT, PALMER, and TPSS) added in later stages, may be found under other data sources. This occurs mainly for BATS and CALCOFI, which have their earlier chlorophyll data in SeaBASS with metadata strings “seabass_bats*” and “seabass_cal*”, and CIMT which has some of its data under COASTCOLOUR. After all data from a given source were free of duplicates, they were merged consecutively by variable in the final table. During this process, we also searched for rows (stations) that were separated from each other by time differences less than 5 min and horizontal spatial differences of less than 200 m. When such rows were found, the observations in those rows were merged into a single row. The compiled merged data were compared with the original sets to certify that no errors occurred during the merging. As a final step, a water-column (station) depth was recorded for each observation, which was the closest water-column depth from the ETOPO1 global relief model (National Geophysical Data Center ETOPO1; Amante and Eakins, 2009). For observations where the closest water depth was above sea level (e.g. data collected very near the coast), it was given the value of zero.

Table 2 Original sets of data and data contributors in the final table.

Data processing thus included two major steps: pre-processing and merging. The first step was related to the processing of each of the 27 contributing data sets and aimed to identify problems and convert the data of interest to a standard format. The second step dealt with the integration of all the contributing sets of data into a unified data set and included the elimination of duplicated data between the individual sets of data. In the next subsections, a brief overview of each original set of data is provided.

2.2 Pre-processing of each set of data

2.2.1 Marine Optical BuoY (MOBY)

MOBY is a fixed mooring system operated by the National Oceanic and Atmospheric Administration (NOAA) that provides a continuous time series of water-leaving radiance and surface irradiance in the visible region of the spectra since 1997. The site is located a few kilometres west of the Hawaiian Island of Lanai where the water depth is about 1200 m. Since its deployment, MOBY measurements have been the primary basis for the on-orbit vicarious calibrations of the SeaWiFS and MODIS ocean colour sensors. A full description of the MOBY system and processing is provided in Clark et al. (2003). Data are freely available for scientific use at the MOBY Gold directory. The products of interest are the “scientific time series” files, which refer to MOBY data averaged over sensor-specific wavelengths and particular hours of the day (around 20:00–23:00 UTC). For this work, the satellite band-average products for SeaWiFS, MODIS AQUA, MERIS, VIIRS-SNPP, VIIRS-JPSS (also known as NOAA-20 VIIRS), OLCI-S3A, and OLCI-S3B were compiled from the “R2017 reprocessing”. The “inband” average subproduct was used, and to maintain the highest quality, only data determined from the upper two arms (“Lw1”) and flagged “good” quality were acquired. Data from the MOBY203 deployment were discarded due to the absence of surface irradiance data. The compiled variable was the remote-sensing reflectance, “rrs”, which was computed from the original water-leaving radiance (“Lw”) and surface irradiance (“Es”). The water-leaving radiances were corrected for the bidirectional nature of the light field (Morel and Gentili, 1996; Morel et al., 2002) using the same lookup table and method as that used in the SeaWiFS Data Analysis System (SeaDAS) processing code. The MOBY data were reprocessed in 2017 (“MOBY R2017 reprocessing”) to include various improvements in the calibration of the instrument and post-processing, which include: (1) a new method to extrapolate the upwelling radiance attenuation coefficient to the surface (Voss et al., 2017); (2) an increase in arm depth by 0.234 m; and (3) a single pixel shift in the data for the red spectrograph collected at a bin factor of 384. Only the last two changes were included in the present compilation. The first change uses model results to improve Lw at wavelengths above 575 nm, by correcting the diffuse upwelling radiance attenuation coefficient for inelastic effects. Thus for wavelengths above 575 nm, the Lw21 product in the Gold directory should be investigated. As mentioned before, the MOBY data compiled in this work are sensor-specific. Therefore, attention is necessary to use the correct MOBY data when validating a particular sensor. The way MOBY data are stored in the final merged table is consistent with the original wavelengths; however, these wavelengths can differ from what is sometimes expected to be the central wavelength of a given band and sensor. Irrespective of the wavelength where MOBY data are stored in the final table, for validation of bands 1–6 of SeaWiFS, MOBY data stored in the final merged table at 412, 443, 490, 510, 555, and 670 nm, respectively, should be used. For validation of bands 1–7 of MODIS AQUA, MOBY data stored in the final merged table at 416, 442, 489, 530, 547, 665, and 677 nm, respectively, should be used. For validation of bands 1-10 of MERIS, MOBY data stored in the final merged table at 410.5, 440.4, 487.8, 507.7, 557.6, 617.5, 662.4, 679.9, 706.2, and 752.5 nm, respectively, are the appropriate data. For validation of bands 1–12 of OLCI-S3A, MOBY data stored in the final merged table at 400.3032, 411.8453, 442.9626, 490.493, 510.4676, 560.4503, 620.4092, 665.2744, 674.0251, 681.5705, 709.1149, and 754.1813, respectively, are the appropriate data. For validation of bands 1–12 of OLCI-S3B, MOBY data stored in the final merged table at 400.5947, 411.9509, 442.9882, 490.3991, 510.4022, 560.3664, 620.284, 665.1312, 673.8682, 681.3856, 708.9821, and 754.0284, respectively, are the appropriate data. For validation of bands 1–5 of VIIRS-SNPP, MOBY data stored in the final merged table at 412.9, 444.5, 481.2, 556.3, and 674.6 nm, respectively, are the appropriate data. Finally, for validation of bands 1–5 of VIIRS-JSPP, MOBY data stored in the final merged table at 411, 445, 489.01, 556, and 667 nm, respectively, are the appropriate data. For the latter sensor, the original value was 489 nm, but it was changed to 489.01 nm to differentiate from the 489 nm of MODIS AQUA. The lookup table to fully normalize “rrs” only covers the range 413–660 nm; compared to the previous versions of the compilation, in the present version, the “rrs” MOBY at wavelengths outside this range were not discarded and fully normalized using the closest entry of the lookup table (i.e. at 413 or 660 nm).

2.2.2 BOUée pour l'acquiSition de Séries Optiques à Long termE (BOUSSOLE)

BOUSSOLE project started in 2001 with the objective of establishing a time series of bio-optical properties in oceanic waters to support the calibration and validation of ocean colour satellite sensors (Antoine et al., 2006). The project consists of a monthly cruise program and a permanent optical mooring (Antoine et al., 2008). The mooring collects radiometry and inherent optical properties (IOPs) in continuous mode every 15 min at two depths (4 and 9 m nominally). The monthly cruises are devoted to the mooring servicing, to the collection of vertical profiles of radiometry and IOPs, and to water sampling at 11 depths from the surface down to 200 m, for subsequent analyses including phytoplankton pigments, particulate absorption, CDOM absorption, and suspended particulate matter load. The BOUSSOLE mooring is in the western Mediterranean Sea at a water depth of 2400 m. All pigment (2001–2019) and radiometric (two subsets: 2003–2012 and 2015–2019) data were provided by the Principal Investigators. The first radiometric subset was obtained from measurements made with multispectral Satlantic OCI-200 radiometers; the second radiometric subset was obtained from measurements made with hyperspectral Satlantic OCR radiometers, convolved with spectral response function of Sentinel3 OLCI-A bands. The compiled variables were “rrs” and “chla_hplc”. Remote-sensing reflectance was computed from the original “fully-normalized” water-leaving radiance (“nLw_ex”), which is the “normalized” water-leaving radiance (“nLw” previously described), with a correction for the bidirectional nature of the light field (Morel and Gentili, 1996; Morel et al., 2002). The solar irradiance (“Fo”) was computed from two available variables in the original set of data: the normalized water-leaving radiance (“nLw”) and the remote-sensing reflectance (“rrs”), using the equation “Fo = nLw/rrs”. Only radiometric observations that meet the following criteria were used: (1) tilt of the buoy was less than 10^∘; (2) the buoy was not lowered by more than 2 m as compared to its nominal water line (to ensure the Es reference sensor is above water and exempt from sea spray); and (3) the solar irradiance was within 10 % of its theoretical clear-sky value (determined from Gregg and Carder, 1990). The latter criterion was used to select clear skies only. An additional quality control was to remove observations that were 50 % higher or lower than the daily average. This removed a small number of “spikes” in the time series. The final quality control step was to remove days where the standard deviation was more than half of the daily average. This was meant to identify days with high variability. Very few days (N=2) were removed with this test. These quality control criteria were applied per wavelength, which resulted in some observations with an incomplete spectrum.

2.2.3 AErosol RObotic NETwork-Ocean Color (AERONET-OC)

AERONET-OC is a component of AERONET, including sites where sun photometers operate with a modified measurement protocol leading to the determination of the fully-normalized water-leaving radiance (Zibordi et al., 2006, 2009). As a result of a collaboration between the Joint Research Centre (JRC) and NASA to develop (Hooker et al., 2000) and exploit (Zibordi et al., 2002) the technology, this component has been specifically developed for the validation of ocean colour radiometric products. The strength of AERONET-OC is “the production of standardized measurements that are performed at different sites with identical measuring systems and protocols, calibrated using a single reference source and method, and processed with the same codes” (Zibordi et al., 2006, 2009). All high quality data (“Level-2”) were acquired from the project website for 11 sites: Abu_Al_Bukhoosh (∼25^∘ N, ∼53^∘ E), COVE_SEAPRISM (∼36^∘ N, ∼75^∘ W), Gloria (∼44^∘ N, ∼29^∘ E), Gustav_Dalen_Tower (∼58^∘ N, ∼17^∘ E), Helsinki Lighthouse (∼59^∘ N, ∼24^∘ E), LISCO (∼40^∘ N, ∼73^∘ W), Lucinda (∼18^∘ S, ∼146^∘ E), MVCO (∼41^∘ N, ∼70^∘ W), Palgrunden (∼58^∘ N, ∼13^∘ E; Philipson et al., 2016), Venice (∼45^∘ N, ∼12^∘ E), and WaveCIS_Site_ CSI_6 (∼28^∘ N, ∼90^∘ W). The compiled variable was “rrs”. Remote-sensing reflectance was computed from the original “fully-normalized” water-leaving radiance (see Sect. 2.2.2 for definition). The solar irradiance (“Fo”), which is not part of the AERONET-OC data, was computed from the Thuillier et al. (2003) solar spectrum irradiance by averaging “Fo” over a wavelength-centred 10 nm window. Data were compiled for the exact wavelengths of each record, which can change over time for a given site depending on the specific instrument deployed. In comparison with the previous version of the compilation, the present OC-CCI data set version 3, now uses the “version 3” reprocessing of AERONET-OC data (Zibordi et al., 2021).

2.2.4 SeaWiFS Bio-optical Archive and Storage System (SeaBASS)

SeaBASS is one of the largest archives of in situ marine bio-optical data (Werdell et al., 2003) with a long-established inventory (Hooker et al., 1994). It is maintained by NASA's Ocean Biology Processing Group (OBPG) and includes measurements of optical properties, phytoplankton pigment concentrations, and other related oceanographic and atmospheric data. The SeaBASS database consists of in situ data from multiple contributors, collected using a variety of measurement instruments with consistent, community-vetted protocols, from several marine platforms such as fixed buoys, handheld radiometers, and profiling instruments. Quality control of the received data includes a rigorous series of protocols that range from file format verification to inspection of the geophysical data values (Werdell et al., 2003). Radiometric data were mostly acquired through the “validation” search tool, which provided in situ data with matchups for particular ocean colour sensors (Bailey and Werdell, 2006). The criterion in the search–query was defined to have the minimal flag conditions in the satellite data, to retrieve a greater number of matchups, and therefore in situ data. Regarding phytoplankton pigment data, the majority were acquired through the “pigment” search tool, which provided pigment data directly from the archives. As was stated in the SeaBASS website, the “pigment” search tool was originally designed to return only in vitro fluorometric measurements, which is consistent with our approach, but over time chlorophyll-a measurements made using other methods (e.g. in vivo fluorometry) were included in the retrieved pigment data. In the pigment data used in this work, a large number of in situ fluorometric measurements from continuous underway instruments were identified and discarded. These data were initially identified from cruises with more than 50 observations per day and then re-checked in the SeaBASS website to confirm whether indeed they were continuous underway measurements. A total of 120 412 such measurements were identified and discarded. Given the large volume of this group of data, it is possible that some chlorophyll-a observations from in vivo methods may have escaped the scrutiny and persisted into the final merged data set. The “pigment” search tool was recently discontinued, and instead the “file” search tool can be used, which was also used here to acquire chlorophyll, as well as radiometric observations, for more recent years. The remote sensing reflectance acquired from the “file” search tool was corrected for the bidirectional effects (Morel and Gentili, 1996; Morel et al., 2002). The compiled variables from SeaBASS data were “rrs”, “chla_hplc”, “chla_fluor”, “aph”, “adg”, “bbp”, and “kd”.

2.2.5 NASA bio-Optical Marine Algorithm Data set (NOMAD)

NOMAD is a publicly-available data set compiled by the NASA OBPG at the Goddard Space Flight Center. It is a high-quality global data set of coincident radiometric and phytoplankton pigment observations for use in ocean colour algorithm development and satellite-data product-validation activities (Werdell and Bailey, 2005). The source bio-optical data is the SeaBASS archive; therefore, many dependencies exist between these two data sets, which were addressed during the merging. The current version (Version 2.0 ALPHA, 2008) includes data from 1991 to 2007 and an additional set of observations of inherent optical properties. The current version was used in this work, but with an additional set of columns of remote-sensing reflectance corrected for the bidirectional effects (Morel and Gentili, 1996; Morel et al., 2002). This additional set of columns was provided directly by the NOMAD creators. The compiled variables were “rrs”, “chla_hplc”, “chla_fluor”, “aph”, “adg”, “bbp”, and “kd”. Conversion was necessary only for “aph”, “adg”, and “bbp”, and followed the procedures described in Sect. 2.1. For the calculation of “bbp”, the variable “bb” was used with a smooth fitting to remove noise. A portion of NOMAD data were optically weighted (for methods see Werdell and Bailey, 2005). These data are not consistent with the protocols chosen in this work, but these observations were retained since NOMAD is a widely-used data set in ocean colour validation.

2.2.6 MERIS Match-up In situ Database (MERMAID)

MERMAID provides in situ bio-optical data matched with concurrent and comparable MERIS Level 2 satellite ocean colour products (Barker, 2013a, b). The MERMAID in situ database consists of data from multiple contributors, measured using a variety of instruments and protocols, from several marine platforms such as fixed buoys, handheld radiometers, and profiling instruments. Comprehensive quality control and protocols are used by MERMAID to integrate all the data into a common and comparable format (Barker, 2013a, b). Access to MERMAID data is limited to the MERIS Validation Team, the MERIS Quality Working Group, and to the in situ data contributors. For this work, access has been granted to the MERMAID database through a signed Service Level Agreement. The MERMAID data include sub-sets of several data sets used in this compilation (MOBY, AERONET-OC, BOUSSOLE, NOMAD). These observations were removed from the MERMAID data set to avoid duplication (as discussed in Sect. 2.1). The compiled variables were “rrs”, “chla_hplc”, “chla_fluor”, “aph”, “adg”, “bbp”, “kd”, and “tsm”. Remote-sensing reflectance was calculated by dividing by π the original “fully-normalized” water-leaving reflectance (“Rw_ex”), which is the water-leaving reflectance (Rw =π Lw $/$ Es), with a correction for the bidirectional nature of the light field (Morel and Gentili, 1996; Morel et al., 2002). Conversion was also necessary for “aph”, “adg”, and “bbp” and followed the procedures described in Sect. 2.1.

In comparison with the previous version of the compilation, a set of “chla_fluor” observations from MERMAID were considered suspicious and excluded from the compilation (N=3241, from mermaid_MAREL-carnot, mermaid_MAREL-itroise, and mermaid_MAREL-vilaine).

2.2.7 Hawaii Ocean Time-series (HOT)

HOT programme provides repeated comprehensive observations of the hydrography, chemistry, and biology of the water column at a station located 100 km north of Oahu, Hawaii, since October 1988 (Karl and Michaels, 1996). This site is representative of the North Pacific subtropical gyre. Cruises are made approximately once a month to the deep-water Station ALOHA (A Long-Term Oligotrophic Habitat Assessment; 22^∘45^′ N, 158^∘00^′ W). Pigment data (“chla_hplc” and “chla_fluor”) were extracted directly from the project website. Radiometric measurements from the HOT project are also available, but observations of “rrs” and “kd” from the HOT project were acquired in this work as part of the SeaBASS data set.

2.2.8 Geochemistry, Phytoplankton, and Color of the Ocean (GeP&CO)

GeP&CO is part of the French PROOF programme and aims to describe and understand the variability of phytoplankton populations and to assess its consequences on the geochemistry of the oceans (Dandonneau and Niang, 2007). It is based on the quarterly travels of the merchant ship Contship London from France to New Caledonia in the Pacific. A scientific observer sailed on each trip and operated the sampling for surface water, filtration, various measurements, and checking at several times of each day. The experiment started in October 1999 and finished in July 2002. Pigment data were extracted from the project website. Additional pigment data obtained during the OISO-4 cruise in the south Indian Ocean onboard R/V Marion-Dufresne (January–February 2000) were added. The samples were measured by Yves Dandonneau following the method used in the GeP&CO project. The compiled variable was “chla_hplc” and “chla_fluor”.

2.2.9 Atlantic Meridional Transect (AMT)

AMT is a multidisciplinary programme which undertakes biological, chemical, and physical oceanographic research during transects between the UK and destinations in the South Atlantic (Robinson et al., 2006). The programme was established in 1995 (e.g. Robins et al., 1996; Aiken et al., 1998) and since then has completed 29 research cruises. Pigment data between 1997 (AMT5) and 2018 (AMT28) were mostly provided by the British Oceanographic Data Centre (BODC) following a specific request for discrete observations of chlorophyll-a concentration since 1997. The AMT data were isolated by searching for the string “AMT” in the “cruise” columns and the respective Principal Investigators were then searched individually in a separated metadata file. Data not flagged with highest quality or without method of measurement were not used. For any interest in the original data, BODC is the point of contact, which ensures that if there are any updates, the most recent data are supplied. In the case of AMT 26, 27, and 28, data were provided to the OC-CCI project by Gavin Tilstone, whereas in the case of AMT 20 and 23, data were provided by Robert J. W. Brewin. The compiled variables are “chla_hplc” and “chla_fluor”.

2.2.10 International Council for the Exploration of the Sea (ICES)

ICES is a network of more than 4000 scientists from almost 300 institutes, with 1600 scientists participating in activities annually. The ICES Data Centre manages a number of large data set collections related to the marine environment covering the Northeast Atlantic, Baltic Sea, Greenland Sea, and Norwegian Sea. Most of data originate from national institutes that are part of the ICES network of member countries. Data were provided (on 28 April 2014) from the ICES database on the marine environment (Copenhagen, Denmark) following a specific request. The ICES data were made available under the ICES data policy, and if there is any conflict between this and the policy adopted by the users, then the ICES policy applies. The compiled variables were “chla_hplc” and “chla_fluor”.

2.2.11 Arctic System Science Primary Production (ARCSSPP)

ARCSSPP database is a synthesis of observations between 1954 and 2006 from the Arctic Ocean and northern seas (Matrai et al., 2013). The observations were acquired from data repositories, publications or provided by individual investigators. The database includes quality-controlled observations of productivity and chlorophyll-a, photosynthetically available radiation, and hydrographic parameters. This collection of data was acquired at http://www.nodc.noaa.gov/cgi-bin/OAS/prd/accession/download/63065 (last access: 18 December 2022). For the present work, only observations of chlorophyll-a concentration with known time zones were used. The compiled chlorophyll observations were from discrete samples, but the exact method (either “chla_fluor” or “chla_hplc”) was not available for all observations. Thus, the ARCSSPP chlorophyll observations were marked as “chla_fluor”, although some might have been from HPLC measurements and were flagged with “1” in a column “flag_chla_method”. The compiled variable was “chla_fluor”.

2.2.12 Data provided by Astrid Bracher, Alfred-Wegener-Institute Helmholtz Centre for Polar and Marine Research (AWI)

In this work, the AWI data source refers to the group of observations that were provided to the OC-CCI project by Astrid Bracher. These are bio-optical observations collected during several cruises across the globe. All data were available through the PANGAEA repository. Observations of concentration of chlorophyll-a, 1 nm spectrally resolved remote sensing reflectances, and algal pigment absorption coefficient were considered. The methods for these observations are described by Taylor et al. (2011a), Liu et al. (2018a), and Tilstone et al. (2020). For chlorophyll, data from the following cruises were used: ANT-XXIV/1, ANT-XXIV/4, ANT-XXVI/4, and MSM18/3 (Bracher et al., 2015a); SO202/2 (Zindler et al., 2013a); ANT-XXVII/2 (Bracher, 2015); ANT-XXV/1 (Taylor et al., 2011b); ANT-XXVIII/3 and SO218 (Soppa et al., 2014); ANT23-1 (Bracher et al., 2015b); MSM9-1 (Bracher et al., 2017); M91 (Hepach et al., 2016); SO234+235 (Bracher et al., 2019); SO243 (Bracher, 2019a); PS93.2 (Liu et al., 2018b); HE462 (Bracher and Wiegmann, 2019); PS99.1 (Liu et al., 2019a); PS99.2 (Liu et al., 2018c); PS103 (Bracher, 2019b); PS107 (Liu et al., 2018d); and PS113 (Bracher et al., 2020). Concerning remote sensing reflectances, the observations taken during cruises ANT-XXIV/4 and ANT-XXVI/4 (Bracher et al., 2015a), ANT-XXV/1 (Taylor et al., 2011b), and ARK26-3 (Bracher et al., 2018) were gathered. The remote sensing reflectances were corrected for the bidirectional nature of the light field (Morel and Gentili, 1996; Morel et al., 2002). The absorption coefficients were taken during cruises SO202/2 (Zindler et al., 2013), ANT_XXV/1 (Taylor et al., 2011b), ANT-XXVI/3 and ANT-XXVIII/3 (Soppa et al., 2013), ARK26-3 (Gonçalves-Araujo et al., 2018), PS93.2 (Wiegmann et al., 2019), PS99.2 (Liu et al., 2019b), and PS107 (Liu et al., 2019c). The compiled variables were “chla_hplc”, “rrs”, and “aph”.

2.2.13 Bermuda Atlantic Time-series Study (BATS)

BATS is a long-term study by the Bermuda Institute of Ocean Sciences based on regular cruises in the western Atlantic Ocean (Sargasso Sea) since 1988. The cruises at BATS site (∼31^∘40^′ N, 64^∘10^′ W) sample ocean temperature and salinity, but are focused on biogeochemical variables such as nutrients, dissolved inorganic carbon, oxygen, HPLC of pigments, primary production, and sediment trap flux. In this work, all the phytoplankton pigment data available from the BATS website (http://bats.bios.edu/bats-data/, last access: 18 December 2022) were considered, which also included regional and transect cruises not specific to the nominal BATS site. The compiled variables were “chla_hplc” and “chla_fluor”.

2.2.14 Data provided by Knut Yngve Børsheim (BARENTSSEA)

The BARENTSSEA data source refers to a group of observations that were provided to OC-CCI project by Knut Yngve Børsheim. This collection was developed using data from the archives of the Institute of Marine Research (Norway). It comprises observations of temperature, salinity and chlorophyll-a routinely collected by cruises, mainly in the North Sea, the Norwegian Sea, and the Barents Sea between 1997 and 2013. The chlorophyll-a concentration was measured by filtering and extraction using Turner fluorometers. The compiled variable was “chla_fluor”.

2.2.15 The Fisheries and Oceans Canada database for biological and chemical data (BIOCHEM)

BioChem is an archive of marine biological and chemical data maintained by Fisheries and Oceans Canada (DFO, 2018; Devine et al., 2014). The available observations are from department research initiatives and collected in areas of Canadian interest. Available parameters include pH, nutrients, chlorophyll, dissolved oxygen, and other plankton data (species and biomass). Chlorophyll measurements from in vitro fluorometric methods were acquired (from http://www.dfo-mpo.gc.ca/science/data-donnees/biochem/index-eng.html, last access: 18 December 2022) with close guidance by the BioChem helpdesk, confirming quality and methods. The used data span from 1997 to 2014 and were mainly from the Gulf of St. Lawrence (western North Atlantic). The compiled variable was “chla_fluor”.

2.2.16 British Oceanographic Data Centre (BODC)

BODC is the designated marine science data centre for the United Kingdom. The data used in this work derive from a specific request for discrete observations of chlorophyll-a concentration since 1997. Initially, this request was used to compile AMT data (see Sect. 2.2.9). The remaining data comprising observations of chlorophyll-a concentration from fluorometric and HPLC methods, mostly sampled in the North Atlantic, were analysed and added (the “dataset” string for this data source is “bodc”). Data not flagged with highest quality or without method of measurement were discarded. The compiled variables were “chla_hplc” and “chla_fluor”.

2.2.17 California Cooperative Oceanic Fisheries Investigations (CALCOFI)

CalCOFI is a partnership of the California Department of Fish & Wildlife, National Oceanic & Atmospheric Administration Fisheries Service, and Scripps Institution of Oceanography. CalCOFI has conducted quarterly cruises off southern and central California since 1949. Data collected in the upper 500 m include temperature, salinity, oxygen, nutrients, chlorophyll, primary productivity, plankton biodiversity, and biomass. For this work, only observations of chlorophyll-a concentration derived from fluorometric methods flagged with highest quality were used. Data were acquired from the file “CalCOFI_Database_194903-201911_csv_10Jul2020.zip”. The compiled variable was “chla_fluor”.

Figure 1 Relative spectral frequency of remote-sensing reflectance in the final table, using 10 nm wide class intervals, defined as the ratio of the number of observations at a particular waveband to the total number of observations at all wavebands, multiplied by 100 to report results in percentage. Data at a total of 951 unique wavelengths, between 313 and 1022.1 nm, were compiled.

2.2.18 California Current Ecosystem Long-Term Ecological Research (CCELTER)

CCELTER investigates the California Current coastal pelagic ecosystem, with a focus on long-term forcing. The CCELTER data include primary and derived measurements from both Process and CalCOFI-augmented cruises, as well other time series. CCELTER data include variables from the physical environment, biogeochemistry, and biological populations/communities. For this work, chlorophyll observations measured from discrete bottle samples from CCELTER Process cruises determined by extraction and bench fluorometry (California Current Ecosystem LTER and Goericke, 2020) were used. The compiled variable was “chla_fluor”.

Figure 2 The distribution of (a) “rrs” at 44X nm and (b) “rrs” at 55X nm. Data were first searched at 445 and 555 nm, and then with a search window of up to 8 nm, to include data at 547 nm. The black boxes delimit the percentiles 0.25 and 0.75 of the data and the black horizontal lines show the extension of up to percentiles 0.05 and 0.95. The red line represents the median value and the black circles the values below (and above) the percentile 0.05 (0.95). The number of measurements of each data set is reported on the right axis of the graph.

Figure 3 Temporal distribution of chlorophyll-a concentration (“chl”), remote-sensing reflectance (“rrs”), algal pigment absorption coefficient (“aph”), detrital plus CDOM absorption coefficient (“adg”), particle backscattering coefficient (“bbp”), the diffuse attenuation coefficient for downward irradiance (“kd”), and total suspended matter (“tsm”) in the final table. All chlorophyll data were considered, but for a given station, HPLC data were selected if available. Colours indicate the number of stations available for each variable as a function of month and hemisphere of data acquisition (“N” – Northern Hemisphere; “S” – Southern Hemisphere). The empty (white) squares indicate no data for that month.

2.2.19 Center for Integrated Marine Technologies (CIMT)

CIMT was a non-operational program where marine scientists from different disciplines and institutions combine their efforts on observations directed towards understanding the central California upwelling system. The CIMT archived data include coastal ocean observations from satellites, shipboard data, moorings, and large marine animal movements. For this work, pigment data from discrete bottle samples taken during CIMT monthly cruises were used. Data were acquired from the project website (https://cimt.ucsc.edu/data_portal.htm, last access: 18 December 2022). The compiled variable was “chla_fluor”.

2.2.20 CoastColour Round Robin (COASTCOLOUR)

COASTCOLOUR data sets were designed to evaluate the performance of ocean colour satellite algorithms in the retrieval of water quality parameters in coastal waters (Nechad et al., 2015a). Three types of COASTCOLOUR data sets are available: (1) a match-up data set where in situ bio-optical observations are available simultaneously with a cloud-free MERIS product; (2) an in situ reflectance data set where an in situ reflectance is available simultaneously with an in situ measurement of chlorophyll-a concentration and/or total suspended matter; and (3) a simulated data set where reflectances were generated by a radiative transfer model. This work used the match-up data set, which includes most of the in situ measurements, and is available at https://doi.org/10.1594/PANGAEA.841950 (Nechad et al., 2015b). The match-up data set provides optical, biogeochemical, and physical data collections at 17 sites across the globe. From this data set, observations of reflectance, chlorophyll-a, total suspended matter, and IOPs were compiled. The remote sensing reflectances were corrected for the bidirectional nature of the light field (Morel and Gentili, 1996; Morel et al., 2002). The compiled variables were “rrs”, “chla_hplc”, “chla_fluor”, “aph”, “adg”, “bbp”, and “tsm”.

2.2.21 European Station for Time series in the Ocean, Canary Islands (ESTOC)

ESTOC is an open-ocean monitoring site located in the eastern North Atlantic subtropical gyre. ESTOC was initiated in 1991 with particle flux measurements, and in 1994 began standard observations of the water column, in addition to the deployment of a current meter mooring. The core parameters measured at ESTOC include salinity, temperature, current speed, nutrients, chlorophyll, inorganic carbon, particulate organic carbon and nitrogen, and sinking particle flux (Neuer et al., 2007). For this work, measurements of chlorophyll-a concentration from monthly cruises from 1994 to 2011 were used. These data were provided to CCI following a specific request. The time of day was unavailable and was set to 12:00:00 (UTC). These observations were flagged with “1” in column “flag_time”. The compiled variable was “chla_fluor”.

Figure 4 Ranges of remote-sensing reflectance band ratios (412:443 and 490:555) for all data. The points from the NOMAD data set are shown in blue for reference. To maximize the number of ratios per data set, a search window up to 12 nm was used when the four wavelengths (412, 443, 490, 555) were not simultaneously available. The effect of different search windows was negligible in the ratio distribution.

2.2.22 Australia's Integrated Marine Observing System (IMOS)

IMOS is enabled by Australia's National Collaborative Research Infrastructure Strategy (NCRIS) funded by Australian Government. Since 2006, IMOS is operating a wide range of observing equipment throughout the coastal and open ocean around Australia, making all data openly available to the scientific community, other stakeholders, and users. In this work, the IMOS data contribution refers to two data sets. One is a data collection entitled “IMOS National Reference Station (NRS) – Phytoplankton HPLC Pigment Composition Analysis”, which was acquired from the Australian Ocean Data Network portal (https://portal.aodn.org.au, last access: 18 December 2022). This data set comprises of phytoplankton pigment composition measured by HPLC collected with small vessels on monthly basis at nine National Reference Stations as part of the IMOS National Mooring Network. The other chlorophyll-a data set measured by HPLC and fluorometry methods is a subset (2015–2021) of the IMOS Bio-optical Database also available through the AODN portal. This database comprises of a suite of bio-optical parameters from samples collected during research voyages in Australian waters and is used by the IMOS Ocean Colour Sub-Facility to assess the accuracy of satellite ocean colour products in Australian coastal and open ocean waters (Schroeder et al., 2016). The previous data compilations include an earlier subset of HPLC chlorophyll-a concentration from the IMOS Bio-optical Database that was acquired through the SeaBASS archive. These data can be found under “dataset” string “seabass” and Lesley Clementson as data contributor. The compiled variables for IMOS were “chla_hplc” and “chla_fluor”.

Figure 5 Global distribution of remote-sensing reflectance per data set in the final table. The data sources are identified with different colours. Points show locations where at least one observation is available. Crosses show sites from where time series data of remote-sensing reflectance are available.

Figure 6 Comparison of coincident observations of chlorophyll-a concentration derived with different methods (“chla_fluor” and “chla_hplc”). The data were transformed prior to regression analysis to account for their log-normal distribution.

Figure 7 Number of observations per chlorophyll-a concentration acquired with different methods (“chla_fluor” and “chla_hplc”).

2.2.23 MARineEcosytem DATa (MAREDAT)

MAREDAT database is a global assemblage of pigments measured by HPLC (Peloquin et al., 2013a) from combination of 136 independent field data sets, solicited from investigators and databases. The database provides high quality measurements of taxonomic pigments including chlorophyll a and b, 19'-butanoyloxyfucoxanthin, 19'-hexanoyloxyfucoxanthin, alloxanthin, divinyl chlorophyll-a, fucoxanthin, lutein, peridinin, prasinoxanthin, violaxanthin, and zeaxanthin. The database is available through PANGAEA (https://doi.org/10.1594/PANGAEA.793246, Peloquin et al., 2013b). For this work, only measurements of total chlorophyll-a flagged with high quality were used. The time of day was unavailable and was set to 12:00:00 (UTC). These observations were flagged with “1” in column “flag_time”. The compiled variable was “chla_hplc”.

2.2.24 Palmer Station Long-term Ecological Research (PALMER)

PALMER is a monitoring station located in western Antarctic Peninsula. The Palmer station investigates the marine ecology of the Southern Ocean with focus on the pelagic marine ecosystem, including sea ice habitats, regional oceanography, and nesting sites of seabird predators. The PALMER data include measurements of meteorological, oceanographic, sea ice, predators, nutrients and biogeochemistry, pigments, primary production, zooplankton, and microbes parameters. This work used the measurements of chlorophyll analysed by HPLC and fluorometry taken at the Palmer Station (Palmer Station Antarctica LTER et al., 2020a, b) and from the annual cruises off the coast of the Western Antarctica Peninsula (Palmer Station Antarctica LTER et al., 2018, 2020c). The compiled variables were “chla_hplc”, “chla_fluor”.

Figure 8 Global distribution of chlorophyll-a concentration per interval of the observed value. All chlorophyll data were considered, but for a given station, HPLC data were selected if available.

Figure 9 Global distribution of chlorophyll-a concentration per data set in the final table. All chlorophyll data were considered, but for a given station, HPLC data were selected if available. Crosses show sites from where data of chlorophyll are available in a specific geographic location.

2.2.25 SeaDataNet archive (SEADATANET)

SeaDataNet is a Pan-European infrastructure for ocean and marine data management. It aims to develop a standardized system for managing large and diverse data sets collected by oceanographic cruises and automatic observation systems. For this work, discrete chlorophyll-a concentration observations with an “access restriction” set to “academic” and “unrestricted” were acquired from the SeaDataNet platform with guidance from the helpdesk. Only data from the “Institute of Marine Research – Norwegian Marine Data Centre (NMD), Norway”, which comprised most of the acquired data, were used. All chlorophyll observations were from discrete samples measured by fluorometric, spectrophotometric, or HPLC methods, but the exact method was not given. Thus, the observations were marked as “chla_fluor”, although some were possibly from HPLC measurements, and were flagged with “1” in a column “flag_chla_method”. The compiled variables were “chla_fluor”.

2.2.26 Data provided by Trevor Platt and Shubha Sathyendranath (TPSS)

In this work, the TPSS data source refers to a group of observations that were provided to this compilation by Trevor Platt and Shubha Sathyendranath. This is a collection of bio-optical in situ data collected during cruises predominantly in the Northwest Atlantic, but also from the Indian Ocean, South Pacific, and Central Atlantic (see Sathyendranath et al., 2009 for additional details regarding the cruises). It comprises measurements of phytoplankton pigments and algal pigment absorption coefficients. The time of day was unavailable and was set to 12:00:00 (UTC). These observations were flagged with “1” in column “flag_time”. The compiled variables were “chla_hplc”, “chla_fluor”, and “aph”.

2.2.27 Bio-optical data from Tara expeditions (TARA)

The Tara expeditions consist of several cruises around the world, some with durations of several years, designed to study and understand the distribution of planktonic organisms in the world ocean. The discrete observations of remote sensing reflectance and chlorophyll-a concentration from HPLC measurements taken during the Tara “Oceans” (2009–2013) and “Mediterranean” (2014) expeditions were considered in this work. These data were provided to ESA OC-CCI project by Emmanuel Boss and were available in the SeaBASS archive. The remote sensing reflectances were corrected for the bidirectional nature of the light field (Morel and Gentili, 1996; Morel et al., 2002). The compiled variables were “chla_hplc” and “rrs”.

Figure 10 The chlorophyll-a (mg m⁻³) data partitioned into $\mathrm{5}{}^{\circ }\times \mathrm{5}{}^{\circ }$ boxes showing (a) number of observations, (b) average value, and (c) standard deviation in each box. All chlorophyll data were considered, but for a given station, HPLC data were selected if available. In the standard deviation plot, grey colour boxes represent zero standard deviation (i.e. one observation).

3 Results

In this work, several sets of bio-optical in situ data were acquired, homogenized, and merged into a single unified data set. The data set comprises in situ observations between 1997 and 2021, with a global distribution, and includes the following variables: “rrs”, “chla”, “aph”, “adg”, “bbp”, “kd”, and “tsm”. All observations were processed in such a way that they can be compared directly with satellite-derived ocean colour data. The compiled data set corresponds to a table with a total of 151 673 rows and 3458 columns. Each row represents a unique station in space and time, separated from the rest by at least 5 min and 200 m. For each variable at a given station, three metadata strings are provided: “dataset”, “subdataset”, and “contributor”. The columns of the table take the form described in Table 1. The data contributors are indicated in Table 2. Regarding spectral variables, all original wavelengths were preserved, which required many unique wavelengths to be maintained in the database. No band shifting was performed (though some archived data in some data sources may have been merged with nearby wavelengths) and no minimum number of wavelengths per observation was imposed. This allowed further manipulation of the data set for different purposes. In the following paragraphs, the final group of observations is described in terms of each variable and the corresponding contributing data sets; however, it is important to note that the numbers reported here do not reflect the original numbers in each contributing data set, since observations close in time and space were averaged and quality controls were applied. Furthermore, duplicates across contributing data sets were removed (e.g. NOMAD and others, such as MOBY, were removed from MERMAID; also, data of individual projects, such as PALMER and AMT, can be listed under NOMAD). Nevertheless, the reported numbers still give a general view of the contributions from each data set and provide users with valuable information for analysing each set of data separately.

Figure 11 A remote-sensing reflectance maximum band ratio (as defined in text) ([443,490,510]/555 or [443,490,510]/560 if 555 not available) as a function of chlorophyll-a concentration. All chlorophyll data were considered, but for a given station, HPLC data were selected if available. Data within 2 nm of the wavelengths were used. For reference, the solid and dotted lines show the NASA OC4 and OC4E v6 standard algorithms, respectively (https://oceancolor.gsfc.nasa.gov/atbd/chlor_a/, last access: 18 December 2022). The total number of points was 3645, of which 80 % were from NOMAD.

Observations of remote-sensing reflectance are available at 948 unique wavelengths (i.e. columns), between 313 and 1022.1 nm (Fig. 1). In total, there are 68 641 observations (i.e. rows) of remote-sensing reflectance. The total number of observations are partitioned per contributing data sets as follows: AERONET-OC (34 551), BOUSSOLE (22 620), MOBY (6034), NOMAD (3326), MERMAID (895), SeaBASS (730), AWI (71), COASTCOLOUR (307), and TARA (107). Data from AERONET-OC, BOUSSOLE, and MOBY correspond to continuous time series, and hence the higher number of observations. In comparison with the previous version (Valente et al., 2019), which had reflectance data until 2018, the number of stations increased by ∼15 % (i.e. from 59 781 to 68 641). The new data points are mainly from recent years (2019–2021) and from updates of AERONET-OC, BOUSSOLE, MOBY, MERMAID, and AWI. The new data extended the temporal coverage towards more recent years, but the statistical distribution of values and the spatial coverages (discussed below) have essentially remained the same when compared to the previous version (Valente et al., 2019). This is explained by most of the new observations coming from continuous time series at fixed the locations (AERONET-OC, BOUSSOLE, MOBY).

Figure 12 The distribution of: (a) “aph” at 44X nm; (b) “aph” at 55X; (c) “adg” at 44X nm; (d) “adg” at 55X; (e) “bbp” at 44X nm; (f) “bbp” at 55X; (g) “kd” at 44X nm; (h) “kd” at 55X nm. Data were first searched at 445 and 555 nm, and then with a search window up to 8 nm, to include data at 547 nm. The graphical convention is identical to Fig. 2.

The distribution of the remote sensing reflectances at 44X nm and 55X nm is provided in Fig. 2a and b, respectively. Data were first searched at 445 and 555 nm, and then with a search window up to 8 nm, to include also data at 547 nm. Median values at 44X nm ranged from 0.003 m⁻¹ (AERONET-OC) and 0.009 m⁻¹ (MOBY), whereas at 55X nm the median values lie between 0.001 m⁻¹ (AWI) and 0.007 m⁻¹ (COASTCOLOUR). The observations remain unevenly distributed between each month of the year in both hemispheres, with the summer months having higher data representation (Fig. 3). The Northern Hemisphere has also more data than the Southern Hemisphere (Fig. 3). As a quality control indicator, reflectance band ratios were plotted against each other (490:555 versus 412:443, Fig. 4). Most points are within the boundaries of the NOMAD data set, but some scattered points were found. These points were retained to allow further manipulation with different quality control criteria. The geographic distribution of the remote-sensing reflectance stations (Fig. 5) still show a higher number of observations in some coastal regions, such as those of North America and Northern Europe. Away from continental margins, the Atlantic Ocean has the highest density of observations. Best geographic coverage is provided by the NOMAD database. Data from SeaBASS is also well dispersed in space but fewer in number. Data from MERMAID are mainly located along the coasts of Europe, North America, and the central region of the North Atlantic Ocean. The observations from AERONET-OC, BOUSSOLE, COASTCOLOUR, and MOBY are concentrated in specific sites around the world, while AWI data are available for the Atlantic and Arctic Oceans. TARA data are spread across several regions, with highest data density in the Mediterranean Sea.

Figure 13 The distribution of absorption coefficients band ratios: adg(443) $/$ adg(490), adg(412) $/$ adg(443), aph(490) $/$ aph(443) and aph(412) $/$ aph(443). Data within 2 nm of the wavelengths were used. The graphical convention is identical to Fig. 2. The vertical dashed lines show the lower and upper thresholds used for quality control in the IOCCG report 5. The total number of points for “adg” ratios are divided between NOMAD (89 %), COASTCOLOUR (7 %), MERMAID (3 %), and Seabass (1 %). The total number of points for “aph” ratios are divided between NOMAD (28 %), TPSS (23 %), AWI (23 %), COASTCOLOUR (14 %), SeaBASS (10 %), and MERMAID (2 %).

Observations of chlorophyll-a concentration were divided into those measured by fluorometric or spectrophotometric methods (“chla_fluor”), and HPLC methods (“chla_hplc”). A comparison of the two types of measurements, when available at the same station (Fig. 6), shows good agreement (Trees et al., 1985). No data were filtered for this analysis and the good correlation can be explained in part by the quality control measures implemented by the data providers and curators of repositories such as NOMAD and SeaBASS (Werdell and Bailey, 2005). The total number of stations with concurrent observations of “chla_fluor” and “chla_hplc” is 5953, with contributions from SeaBASS (39 %), TPSS (16 %), PALMER (14 %), NOMAD (11 %), BATS (5 %), COASTCOLOUR (4 %), MERMAID (4 %), HOT (4 %), and AMT + GeP&CO + BODC + CCELTER + CALCOFI (3 %). The “chla_fluor” observations are available in 61 317 stations (rows), with values limited to the range between 0.001 to 100 mg m⁻³ (Fig. 7). They are from NOMAD (2350), SeaBASS (18 575), MERMAID (480), ICES (5421), HOT (755), AMT (396), ARCSSPP (189), BARENTSSEA (7188), BATS (356), BIOCHEM (4592), BODC (895), CALCOFI (5396), COASTCOLOUR (3322), CCELTER (468), CIMT (204.), ESTOC (100), GEPCO (56), IMOS (1136), PALMER (3237), SEADATANET (5403), and TPSS (1000). The total number of “chla_hplc” observations is 27 215, ranging from 0.002 to 99.8 mg m⁻³ (Fig. 7), with contributions from NOMAD (1309), SeaBASS (10 257), MERMAID (707), ICES (2994), HOT (222), GeP&CO (1536), BOUSSOLE (577), AMT (1359), AWI (2343), BATS (334), BODC (735), COASTCOLOUR (848), IMOS (340), MAREDAT (1024), PALMER (1525), TPSS (1002), and TARA (161). Compared to the previous version (Valente et al., 2019), the “chla_hplc” observations increased by ∼16 % (23 550 to 27 215). As for the “chla_fluor” observations, they have decreased (from 61 525 to 61 317), which is explained by the added observations (N=3033) being less than the removed stations due to quality control (N=3241; see Sect. 2.2.6). The new data points come from updates of BOUSSOLE, MERMAID, SeaBASS, HOT, AMT, PALMER, CCELTER, CALCOFI, AWI, and IMOS.

The combined chlorophyll data set (all chlorophyll data considered, but for a given station, HPLC data were selected if available), has a total of 82 543 observations, which represents an increase of ∼4 % (i.e. from 79 731 to 82 543) when compared to the previous version (Valente et al., 2019). The present version represents a major increase in the number of recent observations. For the combined chlorophyll data set, 533 stations were available in previous version for the period 2016–2017 (previous version had chlorophyll data until 2017). Now, there are 5140 stations for the period 2016–2021, which represents an increase of ∼964 % for the period of 2016 onwards. Overall, data distribution and spatial coverage remain the same between present and previous versions. Approximately 10 %, 50 %, and 40 % of observations are from oligotrophic (<0.1 mg m⁻³), mesotrophic (0.1–1 mg m⁻³), and eutrophic (>1 mg m⁻³) waters, respectively. When compared with the proportions of the world ocean in these trophic classes, 56 % oligotrophic, 42 % mesotrophic, and 2 % eutrophic (Antoine et al., 1996), oligotrophic waters are still under-represented relative to eutrophic waters in the compilation. The combined chlorophyll data set is also still unevenly distributed geographically, with higher coverage in the Northern Hemisphere (Fig. 3). The spatial distribution of the chlorophyll values for the combined data set (Fig. 8) shows a good agreement with known biogeographical features, such as lower chlorophyll values in the subtropical gyres and higher values in temperate, coastal, and upwelling regions. Many regions show a good spatial coverage (e.g. Atlantic and Pacific Ocean), while others are less well sampled (e.g. Southern and Indian oceans). Of the contributing data sets, SeaBASS provides the most extensive global spatial coverage (Fig. 9). Other data sets also provide broad coverage from several locations across the globe (NOMAD, GEPCO, MAREDAT, TARA). The ICES, MERMAID, and BODC data are mainly located along the coastal regions of Europe. The AMT and many AWI data mostly cover the Atlantic Ocean. Other AWI data cover the Amundsen to Bellinghausen Sea of the Southern Ocean, the North Sea, the Arctic Ocean, the Indian Ocean, and the subtropical and tropical Pacific. Coverage for the Arctic region and northern seas of the North Atlantic is provided by SEADATANET, ARCSSPP, and BARENTSEA data sets. Observations from BIOCHEM and TPSS are mostly from the Northwest Atlantic, whereas CALCOFI, CCELTER, and CIMT provide data for the western coast of North America. The data from IMOS mainly covers the coastal Australian waters. The remaining data sets provide observations for fixed locations: PALMER (western Antarctic peninsula), COASTCOLOUR (17 coastal sites across the world), BATS (Bermuda, North Atlantic), BOUSSOLE (Mediterranean), HOT (Hawaii, North Pacific), and ESTOC (Canaries, North Atlantic). Figure 9 shows all data sources that contribute with chlorophyll observations, but many overlap each other, especially around Europe and North America. For additional analysis and as an example of the applications of the compiled dataset, the combined chlorophyll data (“chla_fluor” and “chla_hplc”) were partitioned into $\mathrm{5}{}^{\circ }\times \mathrm{5}{}^{\circ }$ boxes, and for each box the number of observations, average value, and standard deviation were computed (Fig. 10a, b, and c, respectively). The number of observations can be very high (>1000) in some boxes along the European and North American coastlines and relatively low (<20) in oceanic regions. The well-known global biogeographical features, such as the lower chlorophyll in the subtropical gyres and higher values in coastal and upwelling areas, clearly emerge in the average value map (Fig. 10b). There is a close correspondence between the spatial patterns of the average and standard deviation maps (Fig. 10b and c), which may be an indicator of the data quality.

Coincident observations of chlorophyll-a concentration and remote-sensing reflectance are available at 3645 stations. These observations are mostly from NOMAD (80 %), MERMAID (9 %), COASTCOLOUR (6 %), and SeaBASS (3 %). The maximum of three selected band ratios of remote-sensing reflectance is plotted against chlorophyll-a concentration (Fig. 11). The “chla” values used are the combined HPLC and fluorometric chlorophyll-a, and for the “rrs”, the closest spectral observation within 2 nm was used. The maximum band ratios were calculated as the maximum of [rrs(443) $/$ rrs(555), rrs(490) $/$ rrs(555), rrs(510) $/$ rrs(555)] or [rrs(443) $/$ rrs(560), rrs(490) $/$ rrs(560), rrs(510) $/$ rrs(560)] if rrs(555) was not available. The relationship between maximum band ratio and chlorophyll is close to the NASA OC4 and OC4E v6 standard algorithm (http://oceancolor.gsfc.nasa.gov/cms/atbd/chlor_a, last access: 18 December 2022) similarly based on maximum band ratios, providing confidence in the quality of the compiled data. Compared to the previous version (Valente et al., 2019), the relations between maximum band ratio and chlorophyll are not altered by the additional number of concurrent observations (N=13).

Table 3 Summary of median values for “aph”, “adg”, and “bbp” at 44X and 55X nm for each data set (as shown in Fig. 12a–f). Data were first searched at 445 and 555 nm, and then with a search window up to 8 nm, to include data at 547 nm.

Figure 14 Global distribution of observations of inherent optical properties (algal pigment absorption coefficient “aph”, detrital plus CDOM absorption coefficient “adg”, and particle backscattering coefficient “bbp”) in the final table.

Figure 15 Global distribution of diffuse attenuation coefficient for downward irradiance (“kd”) and total suspended matter (“tsm”) per data set in the final table. The “tsm” and “kd” points from MERMAID overlap each other in west Black Sea (∼40^∘ N, 30^∘ E) and Arctic (∼70^∘ N, 120^∘ W).

The inherent optical properties (“aph”, “adg”, and “bbp”) are available at 550 unique wavelengths between 300 and 850 nm. There is a total of 4265, 1654, and 792 observations, for “aph”, “adg”, and “bbp”, respectively. For “aph”, the total number of observations is distributed among NOMAD (1190), TPSS (966), COASTCOLOUR (593), AWI (991), SeaBASS (453), and MERMAID (72). For “adg”, the contributions are as follows: NOMAD (1079), COASTCOLOUR (531), SeaBASS (11), and MERMAID (33). The “bbp” observations come from NOMAD (371), COASTCOLOUR (154), SeaBASS (32), and MERMAID (235). Compared to the previous version (Valente et al., 2019), only “aph” was updated, resulting in a ∼30 % increase (i.e. from 3293 to 4265). Most of the new observations fall within the period 2012–2020, thus increasing the temporal coverage (previous version had “aph” until 2012). Data distribution of “aph”, “adg”, and “bbp” at 44X nm and 55X nm for each data set is provided in Fig. 12a–f. Median values of “aph”, “adg”, and “bbp” at 44X and 55X nm for each data set are summarized in Table 3. As a quality indicator, the following band ratios for the absorption coefficients were calculated: aph(490) $/$ aph(443), aph(412) $/$ aph(443), adg(443) $/$ adg(490), and adg(412) $/$ adg(443). Data within 2 nm of the wavelengths were used to maximize the number of points. The distribution of the ratios is shown in Fig. 13. Several observations were found to be outside the thresholds used in the International Ocean-Colour Coordinating Group (IOCCG) report 5 for quality control (IOCCG, 2006; see dotted vertical black lines in Fig. 13). These points are highlighted here for information but retained in the database, since these were mostly from NOMAD and there was an interest to preserve this data set as a whole. Also, not discarding these data allows further manipulation with different quality control criteria. On the annual scale, the observations of the inherent optical properties continue to be strongly underrepresented in the Southern Hemisphere where there is a complete absence of data during the austral winter (Fig. 3). The new “aph” data in the present version have only increased the spatial coverage in the Arctic region. Overall, the geographic coverage for observations of “aph”, “adg”, and “bbp” (Fig. 14) is poor, with most open ocean regions not being sampled, except for the Atlantic Ocean. Small clusters of data are in specific coastal regions, such as the western coast of North America.

Finally, for the diffuse attenuation coefficient for downward irradiance (“kd”, not updated in present version), there are 25 unique wavelengths between 405 and 709 nm. The total of 2454 observations is divided between NOMAD (2266), SeaBASS (118), and MERMAID (70). Data distribution of “kd” at 44X nm and 55X nm for each data set is shown in Fig. 12g and h. No “kd” data at these wavelengths were available for the SeaBASS data set (only at 490 nm). Median values of “kd” at 44X nm span between 0.08 m⁻¹ (NOMAD) and 0.1 m⁻¹ (MERMAID), whereas at 55X nm the “kd” values are approximately 0.1 m⁻¹ (NOMAD and MERMAID). The best geographical coverage is provided by NOMAD (Fig. 15), with a higher coverage in the Atlantic, compared with other oceans. Except for the coastal regions of North America and the Japan Sea, most coastal regions are not sampled. In the Northern Hemisphere, “kd” is distributed evenly across all months of the year, but in the Southern Hemisphere there are few data points during the austral winter (Fig. 3). For total suspended matter (“tsm”; not updated on present version), there is a total of 1546 observations divided between COASTCOLOUR (1199) and MERMAID (347). The observations of “tsm” are available in a greater number in the Northern Hemisphere (Fig. 3) and are distributed across several coastal regions around Europe, Mediterranean Sea, China Sea, Indonesia, and Australia (Fig. 15).

Although most of the stations with concurrent variables are from the NOMAD data set, for completeness, an examination of bio-optical relationships is provided (Fig. 16). The relation between “aph” at 443 nm and chlorophyll-a (Fig. 16a) agrees with Bricaud et al. (2004). A total of 3387 points exist with these two variables available (29 % from NOMAD, 28 % from TPSS, 22 % from AWI, 10 % from COASTCOLOUR, and remaining 11 % from MERMAID and SeaBASS). The relation between the sum of “aph” and “adg” at 443 nm and “rrs” at 443 nm (Fig. 16b) shows a dispersion similar, except for some scattered points, to an equivalent analysis on the IOCCG report 5 (IOCCG, 2006; see their Fig. 2.3). Again, the scattered data were retained in the final table to preserve the NOMAD data set. A total of 1112 points exists for which these three variables are available (97 % from NOMAD). The relation between the ratio rrs(490) $/$ rrs(555) and kd(490) (Fig. 16c) shows a good agreement with the NASA KD2S standard algorithm (https://oceancolor.gsfc.nasa.gov/atbd/kd/, last access: 18 December 2022). A total of 2280 points exists for which these three variables are available (93 % from NOMAD). The relation between the ratio rrs(490) $/$ rrs(555) and “bbp” at 555 nm (Fig. 16c) shows a good agreement with the relation suggested by Tiwari and Shanmugam (2013). A total of 365 points exists for which these three variables are available (89 % from NOMAD).

Figure 16 Examples of bio-optical relationships in the final merged table: (a) aph(443) versus chlorophyll-a. Total number of points (3387) is divided between AWI (753), COASTCOLOUR (335), MERMAID (214), NOMAD (991), SeaBASS (139), and TPSS (955). For reference, the solid line shows the regression from Bricaud et al. (2004). (b) [aph(443) + adg(443)] versus rrs(443). Total number of points (1112) is divided between MERMAID (33) and NOMAD (1079). (c) [rrs(490) $/$ rrs(555)] versus kd(490). The total number of points (2280) is divided between MERMAID (62), NOMAD (2117), and SeaBASS (101). For reference, the solid line shows the NASA KD2S standard algorithm (https://oceancolor.gsfc.nasa.gov/atbd/kd/, last access: 18 December 2022). (d) [rrs(490) $/$ rrs(555)] versus bbp(555). The total number of points (365) is divided between MERMAID (33), NOMAD (324), COASTCOLOUR (4), and SeaBASS (4). For reference, the solid line shows the relation proposed by Tiwari and Shanmugam (2013). A search window of 2 nm was used for (a) and (b), and a search window of 5 nm was used for (c) and (d) to include data at 560 nm when not available at 555 nm.

4 Data availability

Information about the data availability can be found in Appendix B.

5 Summary and conclusions

In this work, a compilation of bio-optical in situ data is presented, resulting from the acquisition, homogenization, and unification of several sets of data obtained from different sources. The compiled data have a global coverage and span the period from 1997 to 2021, which corresponds to the period of a continuous satellite ocean colour data record. Minimal changes were made on the original data, other than conversion to standard format, data reductions in time and space, and quality control. In situ measurements of the following variables were compiled: remote-sensing reflectance, chlorophyll-a concentration, algal pigment absorption coefficient, detrital and coloured dissolved organic matter absorption coefficient, particle backscattering coefficient, diffuse attenuation coefficient for downward irradiance, and total suspended matter.

The final set of data consists of a substantial number of in situ observations, available in a simple text format, and processed in a way that is used directly for the evaluation of satellite-derived ocean colour data. The major advantages of this compilation are that it merges six commonly-used data sources in ocean colour validation (MOBY, BOUSSOLE, AERONET-OC, SeaBASS, NOMAD, MERMAID), four data sources developed for ocean colour applications (AWI, COASTCOLOUR, TPSS, and TARA), and 17 additional sets of chlorophyll-a concentration data (AMT, ICES, HOT, GeP&CO, ARCSSPP, BARENTSSEA, BATS, BIOCHEM, BODC, CALCOFI, CCELTER, CIMT, ESTOC, IMOS, MAREDAT, PALMER, SEADATANET) free of duplicated observations. This data set was initially created with the intention of evaluating the quality of the satellite ocean colour products from the ESA OC-CCI project, but it can also be used for other purposes, including the validation of retrievals from recent satellite missions such as Landsat 8 and Sentinel 2. It may also be useful in the preparation of future sensors like NASA PACE. In addition, it is likely one of the largest collections of chlorophyll-a concentrations ever assembled, making it useful for the climate and biological scientific communities. The objective of publishing the compilation is to make it easily accessible for the broader community.

In comparison with previous versions, the main advantage of the present version (version 3) is that it includes more recent data (especially from 2016 onwards). These new data are key for the validation of the most recent ocean colour missions (e.g. Sentinel 2B and Sentinel 3B) and for other activities such as System Vicarious Calibration. Future improvements of this data collection could be made by continuing to analyse the available data from the projects, cruises, and archives described in the present work (namely SeaBASS archive which hosts many bio-optical in situ data) and find new data sources, while making sure that the already compiled data sets are the most updated ones following scientific advances and improved quality control measures.

Appendix A: Notation

ad	Detrital absorption coefficient (m−1)
adg	Detrital plus CDOM absorption coefficient (m−1)
AERONET-OC	AErosol RObotic NETwork-Ocean Color
ag	CDOM absorption coefficient (m−1)
AMT	Atlantic Meridional Transect
ap	Particle absorption coefficient (m−1)
aph	Algal pigment absorption coefficient (m−1)
ARCSSPP	Arctic System Science Primary Production
AWI	Data collection from Astrid Bracher
aw	Pure water absorption coefficient (m−1)
BARENTSSEA	Data collection from Knut Yngve Børsheim
BATS	Bermuda Atlantic Time-series Study
bb	Total backscattering coefficient (m−1)
bbp	Particle backscattering coefficient (m−1)
bbw	Backscattering coefficient of seawater (m−1)
BIOCHEM	The Fisheries and Oceans Canada database for biological
	and chemical data
BODC	British Oceanographic Data Centre
BOUSSOLE	Bouée pour l'acquisition d'une Série Optique à Long Terme
CALCOFI	California Cooperative Oceanic Fisheries Investigations
CCELTER	California Current Ecosystem Long-Term Ecological Research
CDOM	Coloured dissolved organic matter
chla	Chlorophyll-a concentration (mg m−3)
chla_fluor	Chlorophyll-a concentration determined from fluorometric
	or spectrophotometric methods (mg m−3)
chla_hplc	Total chlorophyll-a concentration determined from
	HPLC method (mg m−3)
CIMT	Center for Integrated Marine Technology
COASTCOLOUR	Compilation of data in several coastal sites
Es	Surface irradiance (or above-water downwelling irradiance)
	(mW cm−2 µm−1)
ESA	European Space Agency
ESTOC	Estación Europea de Series Temporales del Oceano
Fo	Top-of-the-atmosphere solar irradiance
	(mW cm−2 µm−1)
GeP&CO	Geochemistry, Phytoplankton, and Color of the Ocean
HOT	Hawaii Ocean Time-series
HPLC	High-performance liquid chromatography
ICES	International Council for the Exploration of the Sea
IMOS	Integrated Marine Observing System
kd	Diffuse attenuation coefficient for downward irradiance (m−1)
Lw	Water-leaving radiance (or above-water upwelling radiance)
	(mW cm−2 µm−1 sr−1)
MAREDAT	Compilation of data in several coastal sites
MERIS	Medium Resolution Imaging Spectrometer
MERMAID	MERIS Match-up In situ Database
MOBY	Marine optical buoy
MODIS	Moderate-resolution Imaging Spectroradiometer
MVCO	Martha's Vineyard Coastal Observatory
NASA	National Aeronautics and Space Administration
nLw	Normalized water-leaving radiance (mW cm−2 µm−1 sr−1)
nLw_ex	nLw with a correction for bidirectional effects
	(mW cm−2 µm−1 sr−1)
NOMAD	NASA bio-Optical Marine Algorithm Data set
OC-CCI	Ocean Colour Climate Change Initiative
OLCI	Ocean and Land Colour Instrument
PALMER	Palmer station Long-term Ecological Research
rrs	Remote-sensing reflectance (sr−1)
Rw	Irradiance reflectance (dimensionless)
SeaBASS	SeaWiFS Bio-optical Archive and Storage System
SEADATANET	Archive of in situ marine data
SeaWiFS	Sea-viewing Wide Field-of-view Sensor
TARA	Data collection from global transects
TPSS	Data collection from Trevor Platt and Shubha Sathyendranath
VIIRS	Visible Infrared Imaging Radiometer Suite

Appendix B: Data availability

The compiled data are available at https://doi.org/10.1594/PANGAEA.941318 (Valente et al., 2022). The database is composed of three main tables: table “insitudb_chla.csv” with the observations of “chla_fluor” and “chla_hplc”; table “insitudb_rrs.csv” with observations of “rrs”; and table “insitudb_iopskdtsm.csv” with remaining observations (“aph”, “adg”, “bbp”, “kd”, and “tsm”). The rows within the three tables relate to each other via a unique key (column “idx”). The three tables can be viewed conceptually as one table with all data. To help with data manipulation, six auxiliary tables derived from the previous three main tables are provided. The table “insitudb_metadata.csv” contains all available metadata and helps, for example, to find rows (i.e. “idx”) with multiple variables (e.g. “rrs” and “chla_fluor”). The table “auxiliary_table_contributors.csv” contains the number of observations per data contributor, variable, and dataset. The remaining four tables (“insitudb_rrs_satbands2.csv”, “insitudb_rrs_satbands6.csv”, “insitudb_iopskdtsm_satbands2.csv”, and “insitudb_iopskdtsm_satbands6.csv”) contain the spectral data of the main tables (i.e. “insitudb_rrs.csv” and “insitudb_iopskdtsm.csv”) aggregated within ±2 and ±6 nm, respectively, of SeaWiFS, MODIS AQUA, MERIS, VIIRS-SNPP, VIIRS-JPSS, OLCI-S3A, and OLCI-S3B sensor bands. The tables are generated by assigning, in each row of the main tables (i.e. “insitudb_rrs.csv” and “insitudb_iopskdtsm.csv”), the closest spectral observation within 2 nm (or 6 nm) of a sensor band. The centre wavelengths of each band and sensor used in the generation of the files are the following: SeaWiFS bands 1–8 were centred at [412, 443, 490, 510, 555, 670, 765, 865] nm, respectively; MODIS-AQUA bands 1–9 were centred at [412, 443, 488, 531, 547, 667, 678, 748, 869] nm, respectively; MERIS bands 1–13 were centred at [412, 442, 490, 510, 560, 620, 665, 681, 709, 753, 779, 865, 885] nm, respectively; VIIRS-SNPP bands 1-6 were centred at [410, 443, 486, 551, 671, 746] nm, respectively; VIIRS-JPSS bands 1–6 were centred at [411, 445, 489, 556, 667, 746] nm, respectively; and OLCI-S3A and OLCI-S3B bands 1–15 were centred at [400, 412, 443, 490, 510, 560, 620, 665, 674, 681, 709, 754, 779, 865, 885] nm. An exception to this procedure was made to confirm that the correct MOBY data are stored in the files (see Sect. 2.2.1. for discussion on how MOBY wavelengths are stored in the main file). Finally, a “readme” file is provided to help the user. Table 1 shows what the compiled data looks like. An example of a query for available chlorophyll data from subdataset “seabass_car81” is given.

Table B1 Example of what the compiled data looks like. It shows the result when the compilation is queried for chlorophyll data from subdataset “seabass_car81”.