the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 22 Sep 2023
Enabling FAIR Certification for Micrometeorological Datasets
Stevan Savić
Michael Scriney
Branislava Lalić
Abstract. The current state of weather-induced agricultural losses, water use for irrigation, the appearance of new invasive species and disease vectors, new environmental zoning of plant diseases and pests, deforestation, increased urbanization, rural-to-urban migration and increased urban energy consumption for cooling and heating, together impose a scientific demand for FAIR micrometeorological data. FAIR data and metadata should be easily discoverable by humans or machines, accessible under specific conditions or restrictions, conform to recognized formats and standards to be combined and exchanged, and licensed according to community norms, allowing users to know what kinds of reuse are permitted. However, the lack of FAIR data costs Europe a minimum of €10.2bn per year or approximately 78 % of the Horizon annual 2020 budget. If data met the FAIR principle, it would improve data discovery and access, enable re-use, enhance understanding, especially across domains, reach as many people as possible, be cited more often, and open new routes to build cooperation. To support owners of micrometeorological data to make their data FAIR, the FAIR Micromet Portal was developed within the CA20108 COST Action to guide owners through FAIR principles, in a step-by-step manner, with the ultimate goal of making large volumes of data FAIR. This paper provides a detailed discussion on how this is achieved by validating micrometeorological data stored on the FAIR Micromet Portal against the full set of FAIR metrics.
- Preprint
(1699 KB) - Metadata XML
- BibTeX
- EndNote
Mark Roantree et al.
Status: open (until 08 Dec 2023)
-
RC1: 'Comment on essd-2023-334', Anonymous Referee #1, 14 Nov 2023
reply
The FAIR guiding principles are frequently used by data centres and scientists, but often FAIR is misinterpreted as FA where the main focus is on the first two letters.
The manuscript in question performs a rather comprehensive analysis of the FAIRness of the approach documented. The results of such analyses will often reflect the experience of the person(s) performing the analysis. So also in this context where I would claim that setup documented is pushing the limits and the ambition outlined in the abstract in page 1 at lines 4-6:
FAIR data and metadata should be easily discoverable by humans or machines, accessible under specific conditions or restrictions, conform to recognized formats and standards to be combined and exchanged, and licensed according to community norms, allowing users to know what kinds of reuse are permitted.The setup described will guide interactive reuse of the data, but will be very hard to efficiently reuse in an automated setup where computers retrieve and prepare the data for subsequent analysis/integration. Given the explicit statement in the document on machine interoperability I would expect a more through discussion on requirements and challenges doing so in this specific context.
Even interactive reuse through download of the dataset from Zenodo would be hard to do in a trustworthy manner. Thus I would appreciate a more through discussion of in particular the interoperability aspects, challenges and limitations as well the interaction of this system with other data management systems.
Some specific comments:
1. The reference to WMO documents are made several places, and the reference list includes as its 2 last entries references to WMO documents. These (at least the first) are however not referenced in the document itself where statements like WMO metadata and WMO guide appears.
2. Adding to the previous issue, it would be good to add references to the more modern approach to data sharing within WMO through WMO Information System (WIS) and WMO Integrated Global Observing system (WIGOS) as these are perhaps more relevant for FAIR data exchange today than the document from 2010. A more through analysis of these elements would be welcome, in particular how this solution would integrate with the more modern approach to data management within WMO (now implementing WIS2.0, but WIS has been there since the referenced document from 2010).
3. In page 2, lines 26-30 (and some more around) the language would benefit from cleaning. It is not clear to me what is meant with this statement.
4. Page 2, line 35, a reference to DOI for FAIR is made, I think FAIR requires a PID and not necessarily a DOI (just one way of implementing a PID).
5. Page 2, line 41, a statement is made to the Open Definition, but there is no reference.
6. Page 3, lines 64-71, as climate science is explicitly mentioned, references and discussion of typical climate science repositories and metadata would be beneficial (e.g. ESGF).
7. To the same issue, a more thorough discussion of what metadata is and which types of metadata that exist as well as which standard that are used would be highly beneficial to understand how this system communicates with similar systems holding data and station characteristics. E.g. a better explanation of the distinction between dataset and stations/networks and how these relate would be good.
8. The reference to the WMO standard in page 3 is confusing as there are so many WMO standards in the context of data, observations etc.
9. In page 4, line 77-78 it is stated that "In essence, create a system where researchers can simply "file and forget", content in the knowledge that their data meets FAIR principles and is delivering a greater contribution to science.". This is a statement that require further elaboration. Is there an expectation that climate scientists in general will find and reuse these data documented in this rather heterogeneous manner? It seems the documentation process has focused on the data providers and not necessarily the data consumer. This is an issue that would be good with some reflections over.
10. Page 4, line 87, there is a reference to "FAIR principals". Is this a typo?
11. Page 4, line 96-97, states " What is clear is that some of the metrics and their proposed thresholds are inexact and open to interpretation.". This is an important statement and there are different schools of interpretation, in particular of words like standards, protocols etc especially in the context of achieving data and services that readily talk together. This could be further elaborated and addressed in the discussion.
12. Page 4, line 104, it does not require a DOI, but a DOI will fulfil the requirement...
13. Page 4, line 105, you could establish persistence with your own identifier if you want. There is no more guarantee of such through handle or DOI in the long term.
14. Page 4 line 105->, the discussion on levels would be beneficial to explain further and also to distinguish between metadata describing the data and the stations etc.
15. Page 5 line 108-109 would benefit from a discussion on relations to frameworks like GEOSS, WIS, INSPIRE, Google (schema.org) etc. What is the chance of being discoverable in a system external to this platform?
16. Page 5 line 114, REST are principles for setting up services, but calling it a standard implies that the interface request could be reused in another context. This is possible to achieve with services like OAI-PMH, OGC CSW, OpenSearch, STAC etc, but the chance of 2 implementations using REST serving the same type of data behaving similar is very low. A discussion on this challenge in the FAIR context would be beneficial.
17. Numbering of A* items in section 2.2 should be reviewed as well as analysis of items clarified (e.g. the reference between "A4" and F1.2).
18. Page 6, section 2.3 would need a review and rewrite. I question the addition of I4 (for discovery this should be manageable through F*), concerning use metadata to interpret the data that could be addressed by a more through discussion on formats for data. All the listed formats are open containers that could contain anything and there is no proper reference to knowledge representation like Climate and Forecast Conventions, Darwin Core Archives or other. By following an convention you can make JSON or even CSV "interoperable" but this is not discussed nor explained. A user receiving a CSV file with e.g. temperature would need to know time specifications, how aggregations like mean values are located compared to the time specification, encoding of missing values, units of variables etc. This is at best vaguely discussed and would need more clarification.
19. Page 6, line 130-131 states I2 need a ontology or metamodel. It could be as simple as reusing an existing vocabulary (e.g. CF standard names) and referring to this in the information provided. That would help the user understanding the data. Further explanation is required.
20. Page 6, line 134-136, I would interpret I3 as cross-links (linked data approach) between discovery metadata for datasets, use metadata for interpretation (if not embedded in the data file itself) and station metadata describing features at the station (sensors, surroundings, procedures etc). Thus I find the discussion here a bit confusing since you already have these concepts embedded as different "levels". You mask what you really are doing.
21. Page 6, section 2.3, lacks discussion of machine interaction which is important in the interoperable context.
22. Page 7, section 2.4 a discussion on metadata for different purposes would be necessary. R2 concerning licenses would benefit from a definition of a license and reference to some well tested licenses like Creative Commons, MIT etc relevant for data. R3 is hard to do in a standardised manner today (waiting for PROV-O etc), R4 should relate to standards like CF conventions, WMO formats could be relevant or others for data, but equally a justification why references to standard discovery metadata like ISO-19115 (INSPIRE and WMO), GCMD DIF, EML, etc should be provided as these are both formal and community standards.
23. Page 9 section 3 lacks references to vocabularies etc. Given the issues raised on interpretation of the FAIR principles I have made no direct comments on this section.
24. The NSUNET case study data is hard to interpret since use metadata are so sparse. E.g. is the hourly average centered on the hour, or representing the previous or following hour? What is the time zone of the time variable? I can guess that temperatures are represented in degrees Celsius, but better if stated.Citation: https://doi.org/10.5194/essd-2023-334-RC1 -
RC2: 'Comment on essd-2023-334', Anonymous Referee #2, 27 Nov 2023
reply
1) General Comments
This paper offers a description of how owners of micrometeorological data could make their data FAIR through the FAIR Micromet Portal which was developed within the CA20108 COST Action and provide guidelines on how to make large volumes of data FAIR through validation against the FAIR principles. I think that the methodology described for this micrometric dataset is useful for small volume datasets, but I do not think it can be more widely used.
In particular, much of what is in the Summary starting at Line 145 has been discussed in several other papers that are not cited. I suggest the authors review and cite these in this section. There are many other papers on FAIR assessment methodologies and FAIR maturity matrices that need to be considered.
The paper recommends Zenodo as a source for a DOI. Zenodo is not ideal for, as a generalist repository it does not offer domain-specific curation. Further Zenodo has a limit 50GB per dataset and Zenodo have to be directly contacted support for higher limits. If the authors are offering Zenodo as a solution, then you should also mention this size limit. (see Stall, S., Martone, M. E., Chandramouliswaran, I., Crosas, M., Federer, L., Gautier, J., Hahnel, M., Larkin, J., Lowenberg, D., Pfeiffer, N., Sim, I., Smith, T., Van Gulick, A. E., Walker, E., Wood, J., Zaringhalam, M., & Zigoni, A. (2020). Generalist Repository Comparison Chart. Zenodo. https://doi.org/10.5281/zenodo.3946720 ) Further, clearly this Zenodo solution will not scale for large volume data sets
The dataset itself is of good quality. The methodology of exposing it through the portal is good. I have trouble with the assertions in the paper that this methodology can be used more widely, as thete is already an existing body of literature on this very topic has not been adequately reviewed. It is well known that the FAIR principles are just that: principles, not precise guidelines or specifications and there are many papers around on firstly how these can be changed into guidelines and specifications as well as metrics to assess how compliant a data sets. In addition there have been several papers that describe how to assess “maturity” of a particular implementation of the FAIR principles.
The original paper by Wilkinson et al in 2016 emphasises the need for machine actionable data: I am not convinced that the solution offered is making the metadata and data machine actionable. In particular, the interoperability section does not really make clear how the relevant vocabularies are made FAIR (Principle I2 “(meta)data use vocabularies that follow the FAIR principles”.)
I recommend that this paper be substantially rewritten, with a focus more on the dataset itself and how it is exposed through the portal, with more clarity around how the data within the portal can be made to be findable, accessible, interoperable and reusable by machines.
2) Specific Comments
Line 5: please change “accessible under specific conditions or restrictions” to “openly accessible where possible or if required, accessible under specific conditions or restrictions,”
Line 8: FAIR principles (it is plural)
Line 9: it would “improve discovery, reuse and interoperability”
Line 13: of data FAIR for both humans and machines.
Line 18: Sensors are devices that can detect
Line 26: Adopting the FAIR principles of Wilkinson et al (2016).
Line 32: “accessible under specific conditions or restrictions” to “openly accessible where possible or if required, accessible under specific conditions or restrictions,”
Line 41: Which Open Definition are you referring to – please put in a reference.
Line 64: Change to FAIR-compliant repository
Table 1 – This Table has parallels in a similar system published by Schultes E, Magagna B, Hettne KM, Pergl R, Suchánek M, & Kuhn T (2020). Reusable FAIR Implementation Profiles as Accelerators of FAIR Convergence. In: Grossmann G & Ram S (eds) Advances in Conceptual Modeling. ER 2020. Lecture Notes in Computer Science, vol 12584. Springer, Cham. https://doi.org/10.1007/978-3-030-65847-2_13 They too have extended the original FAIR principles. You should review this paper and reference it. See also Jacobsen, A., de Miranda Azevedo, R., Juty, N., Batista, D., Coles, S., Cornet, R., Courtot, M., Crosas, M., Dumontier, M., Evelo, C. T., Goble, C., Guizzardi, G., Hansen, K. K., Hasnain, A., Hettne, K., Heringa, J., Hooft, R. W. W., Imming, M., Jeffery, K. G., … Schultes, E. (2020). FAIR Principles: Interpretations and Implementation Considerations. Data Intelligence, 2(1–2), 10–29. https://doi.org/10.1162/dint_r_00024.
Note for your F3: Metadata explicitly includes the unique ID – is this the ID of the dataset or of the metadata?
Line 130: To me, Principle I2 states that “(meta)data use vocabularies that follow the FAIR principles”. Many interpret this as meaning that any vocabulary used for either metadata or data should also be Findable, Accessible, Interoperable and Reusable by both humans and machines. For many datasets there are likely to be many vocabularies used that should be available online and can be cited for the I2 principle. It should be clearer in this paper how the vocabularies used in their datasets are themselves made FAIR.
Line 145: This paper should be citing numerous other FAIR assessment methodologies that have been developed for both qualitatively and quantitatively assessing the FAIRness of a dataset. An excellent review of a selection of these is available in Peters-von Gehlen, K, Höck,H, Fast, A, Heydebreck, D, Lammert, A and Thiemann, H. 2022. Recommendations for Discipline-Specific FAIRness Evaluation Derived from Applying an Ensemble of Evaluation Tools. Data Science Journal, 21: 7, pp. 1–21. DOI: https://doi.org/10.5334/dsj- 2022-007 . See also David, R, et al. 2020. FAIRness Literacy: The Achilles’ Heel of Applying FAIR Principles. Data Science Journal, 19: 32, pp. 1–11. DOI: https://doi.org/10.5334/dsj-2020-032 and Bahim, C, et al. 2020. The FAIR Data Maturity Model: An Approach to Harmonise FAIR Assessments. Data Science Journal, 19: 41, pp. 1–7. DOI: https://doi.org/10.5334/dsj-2020-041.
Citation: https://doi.org/10.5194/essd-2023-334-RC2
Mark Roantree et al.
Data sets
Hourly Air Temperature Datasets from city of Novi Sad S. Savic, I. Secerov, J. Dunjic, and D. Milosevic https://doi.org/10.5281/zenodo.7738093
Metadata of the urban meteorological network in Novi Sad (Serbia) S. Savic, I. Secerov, J. Dunjic, and D. Milosevic https://doi.org/10.5281/zenodo.8237900
Mark Roantree et al.
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 199 | 30 | 8 | 237 | 6 | 8 |
- HTML: 199
- PDF: 30
- XML: 8
- Total: 237
- BibTeX: 6
- EndNote: 8
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1