A globally sampled high-resolution hand-labeled validation dataset for evaluating surface water extent maps

Abstract. Effective monitoring of global water resources is increasingly critical due to climate change and population growth. Advancements in remote sensing technology, especially in spatial, spectral, and temporal resolutions, have revolutionized water resource monitoring, leading to more frequent and high-quality surface water extent maps using various techniques such as traditional image processing and machine learning algorithms. However, satellite imagery datasets contain trade-offs that result in inconsistencies in performance. For example, the disparity in measurement principles between optical (Sentinel-2) and radar (Sentinel-1) sensors, and differences in spatial and spectral resolutions among optical sensors. Therefore, developing accurate and robust surface water mapping solutions requires independent validations from multiple datasets in order to identify potential biases within imagery and algorithms. However, high-quality validation datasets are expensive to build, and few contain information on water resources. For this purpose, we introduce a globally sampled, high spatial resolution dataset labeled using 3m PlanetScope imagery. Our surface water extent dataset comprises of 90 images, each with a size of 1024x1024 pixels, which were sampled using a stratified random sampling strategy. We covered all 14 biomes and also highlighted urban and rural regions, lakes, and rivers, including braided rivers and shorelines. To demonstrate the usability of our dataset, we evaluated our novel Sentinel-1 algorithm called the Equal Percent Solution (EPS) for surface water extent delineation. Our method produced an overall accuracy of 88 %, with low commission error. However, EPS also had a high omission error. While investigating the source behind this issue using our hand labels, we found evidence that water signals in Sentinel-1 are affected by turbulence and muddiness. Further, mountainous regions distorted the signals from the water in river valleys leading to inaccuracies. Similar to our evaluation, we expect our dataset to be used for analyzing satellite products and methods to gain insights into their advantages and drawbacks. We expect our high-quality dataset to improve our understanding of the accuracy, spatial generalizability, and robustness of existing surface water products and methods to promote efficient monitoring of our natural resources.