30 Aug 2023
 | 30 Aug 2023
Status: this preprint is currently under review for the journal ESSD.

A globally sampled high-resolution hand-labeled validation dataset for evaluating surface water extent maps

Rohit Mukherjee, Frederick Policelli, Ruixue Wang, Beth Tellman, Prashanti Sharma, Zhijie Zhang, and Jonathan Giezendanner

Abstract. Effective monitoring of global water resources is increasingly critical due to climate change and population growth. Advancements in remote sensing technology, especially in spatial, spectral, and temporal resolutions, have revolutionized water resource monitoring, leading to more frequent and high-quality surface water extent maps using various techniques such as traditional image processing and machine learning algorithms. However, satellite imagery datasets contain trade-offs that result in inconsistencies in performance. For example, the disparity in measurement principles between optical (Sentinel-2) and radar (Sentinel-1) sensors, and differences in spatial and spectral resolutions among optical sensors. Therefore, developing accurate and robust surface water mapping solutions requires independent validations from multiple datasets in order to identify potential biases within imagery and algorithms. However, high-quality validation datasets are expensive to build, and few contain information on water resources. For this purpose, we introduce a globally sampled, high spatial resolution dataset labeled using 3m PlanetScope imagery. Our surface water extent dataset comprises of 90 images, each with a size of 1024x1024 pixels, which were sampled using a stratified random sampling strategy. We covered all 14 biomes and also highlighted urban and rural regions, lakes, and rivers, including braided rivers and shorelines. To demonstrate the usability of our dataset, we evaluated our novel Sentinel-1 algorithm called the Equal Percent Solution (EPS) for surface water extent delineation. Our method produced an overall accuracy of 88 %, with low commission error. However, EPS also had a high omission error. While investigating the source behind this issue using our hand labels, we found evidence that water signals in Sentinel-1 are affected by turbulence and muddiness. Further, mountainous regions distorted the signals from the water in river valleys leading to inaccuracies. Similar to our evaluation, we expect our dataset to be used for analyzing satellite products and methods to gain insights into their advantages and drawbacks. We expect our high-quality dataset to improve our understanding of the accuracy, spatial generalizability, and robustness of existing surface water products and methods to promote efficient monitoring of our natural resources.

Rohit Mukherjee et al.

Status: open (until 06 Oct 2023)

Comment types: AC – author | RC – referee | CC – community | EC – editor | CEC – chief editor | : Report abuse
  • RC1: 'The paper should have a clear focus on the validation dataset (relfecting the title of the paper)', Wolfgang Wagner, 30 Sep 2023 reply

Rohit Mukherjee et al.

Data sets

Global Surface Water Validation Dataset Rohit Mukherjee, Frederick Policelli, Ruixue Wang, Beth Tellman, Prashanti Sharma, Zhijie Zhang, and Jonathan Giezendanner

Rohit Mukherjee et al.


Total article views: 571 (including HTML, PDF, and XML)
HTML PDF XML Total BibTeX EndNote
434 122 15 571 8 11
  • HTML: 434
  • PDF: 122
  • XML: 15
  • Total: 571
  • BibTeX: 8
  • EndNote: 11
Views and downloads (calculated since 30 Aug 2023)
Cumulative views and downloads (calculated since 30 Aug 2023)

Viewed (geographical distribution)

Total article views: 560 (including HTML, PDF, and XML) Thereof 560 with geography defined and 0 with unknown origin.
Country # Views %
  • 1
Latest update: 02 Oct 2023
Short summary
Monitoring global water resources is crucial, but trust in existing solutions remains a challenge. To address this, we present a high-resolution dataset of hand-labeled surface water from 90 diverse locations, evaluating a novel Sentinel-1 algorithm for mapping. Our evaluation provides insights into the advantages and limitations of satellite imagery and methods applied. Our study highlights the need for independent validation datasets to ensure accurate and reliable water resource monitoring.