Articles | Volume 15, issue 1
https://doi.org/10.5194/essd-15-113-2023
© Author(s) 2023. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
https://doi.org/10.5194/essd-15-113-2023
© Author(s) 2023. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Data description article
| 
09 Jan 2023
Data description article |
| 09 Jan 2023
| 09 Jan 2023
MDAS: a new multimodal benchmark dataset for remote sensing
Jingliang Hu
Data Science in Earth Observation (SiPEO), Technical University of Munich (TUM), 80333 Munich, Germany
Rong Liu
Data Science in Earth Observation (SiPEO), Technical University of Munich (TUM), 80333 Munich, Germany
Danfeng Hong
Remote Sensing Technology Institute (IMF), German Aerospace Center (DLR), 82234 Wessling, Germany
now at: Key Laboratory of Computational Optical Imaging Technology, Aerospace Information Research Institute, Chinese Academy of Sciences, 100094 Beijing, China
Andrés Camero
Remote Sensing Technology Institute (IMF), German Aerospace Center (DLR), 82234 Wessling, Germany
Helmholtz AI, 85764 Neuherberg, Germany
Data Science in Earth Observation (SiPEO), Technical University of Munich (TUM), 80333 Munich, Germany
now at: Key Laboratory of Computational Optical Imaging Technology, Aerospace Information Research Institute, Chinese Academy of Sciences, 100094 Beijing, China
Mathias Schneider
Remote Sensing Technology Institute (IMF), German Aerospace Center (DLR), 82234 Wessling, Germany
Franz Kurz
Remote Sensing Technology Institute (IMF), German Aerospace Center (DLR), 82234 Wessling, Germany
Karl Segl
German Research Center for Geosciences (GFZ), Helmholtz Center Potsdam, Telegrafenberg A17, 14473 Potsdam, Germany
Data Science in Earth Observation (SiPEO), Technical University of Munich (TUM), 80333 Munich, Germany
Remote Sensing Technology Institute (IMF), German Aerospace Center (DLR), 82234 Wessling, Germany
Related authors
No articles found.
Nina Merkle, Corentin Henry, Sandeep Kumar Jangir, Jens Hellekes, Felix Rauch, Pablo d'Angelo, and Franz Kurz
ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci., XI-2-2026, 493–501, https://doi.org/10.5194/isprs-annals-XI-2-2026-493-2026, https://doi.org/10.5194/isprs-annals-XI-2-2026-493-2026, 2026
Pablo d'Angelo, Franz Kurz, Alaa Eddine Ben Zekri, and Reza Bahmanyar
ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci., XI-2-2026, 1–8, https://doi.org/10.5194/isprs-annals-XI-2-2026-1-2026, https://doi.org/10.5194/isprs-annals-XI-2-2026-1-2026, 2026
Viola Steidl, Jürgen Kusche, Fupeng Li, and Xiao Xiang Zhu
EGUsphere, https://doi.org/10.5194/egusphere-2026-2312, https://doi.org/10.5194/egusphere-2026-2312, 2026
This preprint is open for discussion and under review for Hydrology and Earth System Sciences (HESS).
Short summary
Short summary
Terrestrial water storage is an indicator of water availability, but forecasting its changes is difficult as it depends on processes acting at varying spatial and temporal scales. We introduce a hierarchical machine learning model that represents the Earth at two scales, the grid scale and the hydrological basin scale, to forecast global water storage changes up to six months ahead. We identify where model skill fades and which challenges to address to improve its forecasts.
Nils Lehmann, Cesar Aybar, Ando Shah, Marcello Passaro, Jonathan L. Bamber, and Xiao Xiang Zhu
Earth Syst. Sci. Data Discuss., https://doi.org/10.5194/essd-2026-232, https://doi.org/10.5194/essd-2026-232, 2026
Preprint under review for ESSD
Short summary
Short summary
We created a global ocean dataset that brings together satellite measurements, model outputs, and observations into one consistent system. We did this to reduce the time and effort needed to combine different data sources, to improve reproducibility, and enable new analyses. The result makes it easier to study ocean changes, compare methods, and support better understanding of climate processes and extreme events.
Yang Mu, Zhitong Xiong, Yi Wang, Muhammad Shahzad, Franz Essl, Holger Kreft, Mark van Kleunen, and Xiao Xiang Zhu
Earth Syst. Sci. Data, 18, 1379–1403, https://doi.org/10.5194/essd-18-1379-2026, https://doi.org/10.5194/essd-18-1379-2026, 2026
Short summary
Short summary
To better protect our planet's forests, we need to know what trees are where. We created GlobalGeoTree, a massive public dataset linking 6.3 million tree locations worldwide with satellite data. This dataset helps computers learn to identify tree species from space, supporting biodiversity monitoring and climate action. Our baseline model shows this is a promising path to understanding global forests.
Xiao Xiang Zhu, Sining Chen, Fahong Zhang, Yilei Shi, and Yuanyuan Wang
Earth Syst. Sci. Data, 17, 6647–6668, https://doi.org/10.5194/essd-17-6647-2025, https://doi.org/10.5194/essd-17-6647-2025, 2025
Short summary
Short summary
We introduce GlobalBuildingAtlas, a publicly available dataset offering global and complete coverage of building polygons (GBA.Polygon), heights (GBA.Height) and Level of Detail 1 3D models (GBA.LoD1). This is the first open dataset to offer high quality, consistent, and complete building data in 2D and 3D at the individual building level on a global scale. With more than 2.75 billion buildings worldwide, it surpasses the most comprehensive database to date by more than 1 billion buildings.
Felix Rauch, Corentin Henry, Manuel Mühlhaus, Franz Kurz, Jens Hellekes, and Nina Merkle
Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XLVIII-4-W16-2025, 83–89, https://doi.org/10.5194/isprs-archives-XLVIII-4-W16-2025-83-2025, https://doi.org/10.5194/isprs-archives-XLVIII-4-W16-2025-83-2025, 2025
Veronika Gstaiger, Claas Köhler, Philipp Hochstaffl, Martin Bachmann, Raquel de los Reyes, Stefanie Holzwarth, Jiaojiao Tian, Peter Gege, Oliver Paxa, Thomas Krauss, Nina Merkle, and Franz Kurz
ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci., X-G-2025, 315–322, https://doi.org/10.5194/isprs-annals-X-G-2025-315-2025, https://doi.org/10.5194/isprs-annals-X-G-2025-315-2025, 2025
Reza Bahmanyar, Jens Hellekes, Manuel Mühlhaus, Veronika Gstaiger, and Franz Kurz
ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci., X-G-2025, 151–158, https://doi.org/10.5194/isprs-annals-X-G-2025-151-2025, https://doi.org/10.5194/isprs-annals-X-G-2025-151-2025, 2025
Franz Kurz, Nina Merkle, Corentin Henry, Reza Bahmanyar, Felix Rauch, Jens Hellekes, Veronika Gstaiger, Dominik Rosenbaum, and Peter Reinartz
Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XLVIII-M-6-2025, 189–195, https://doi.org/10.5194/isprs-archives-XLVIII-M-6-2025-189-2025, https://doi.org/10.5194/isprs-archives-XLVIII-M-6-2025-189-2025, 2025
Zhenghang Yuan, Zhitong Xiong, Lichao Mou, and Xiao Xiang Zhu
Earth Syst. Sci. Data, 17, 1245–1263, https://doi.org/10.5194/essd-17-1245-2025, https://doi.org/10.5194/essd-17-1245-2025, 2025
Short summary
Short summary
ChatEarthNet is an image–text dataset that provides high-quality, detailed natural language descriptions for global-scale satellite data. It consists of 163 488 image-text pairs with captions generated by ChatGPT-3.5 and an additional 10 000 image-text pairs with captions generated by ChatGPT-4V(ision). This dataset has significant potential for training and evaluating vision–language geo-foundation models in remote sensing.
Viola Steidl, Jonathan Louis Bamber, and Xiao Xiang Zhu
The Cryosphere, 19, 645–661, https://doi.org/10.5194/tc-19-645-2025, https://doi.org/10.5194/tc-19-645-2025, 2025
Short summary
Short summary
Glacier ice thickness is difficult to measure directly but is essential for glacier evolution modelling. In this work, we employ a novel approach combining physical knowledge and data-driven machine learning to estimate the ice thickness of multiple glaciers in Spitsbergen, Barentsøya, and Edgeøya in Svalbard. We identify challenges for the physics-aware machine learning model and opportunities for improving the accuracy and physical consistency that would also apply to other geophysical tasks.
Veronika Gstaiger, Nils Machinia, Nina Merkle, Dominik Rosenbaum, Ronald Nippold, Manuel Muehlhaus, Pablo d’Angelo, Corentin Henry, Xiangtian Yuan, Reza Bahmanyar, Franz Kurz, and Christa-Maria Krieg
ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci., X-3-2024, 163–168, https://doi.org/10.5194/isprs-annals-X-3-2024-163-2024, https://doi.org/10.5194/isprs-annals-X-3-2024-163-2024, 2024
Yifan Tian, Yao Sun, and Xiao Xiang Zhu
Abstr. Int. Cartogr. Assoc., 7, 171, https://doi.org/10.5194/ica-abs-7-171-2024, https://doi.org/10.5194/ica-abs-7-171-2024, 2024
Erik Loebel, Mirko Scheinert, Martin Horwath, Angelika Humbert, Julia Sohn, Konrad Heidler, Charlotte Liebezeit, and Xiao Xiang Zhu
The Cryosphere, 18, 3315–3332, https://doi.org/10.5194/tc-18-3315-2024, https://doi.org/10.5194/tc-18-3315-2024, 2024
Short summary
Short summary
Comprehensive datasets of calving-front changes are essential for studying and modeling outlet glaciers. Current records are limited in temporal resolution due to manual delineation. We use deep learning to automatically delineate calving fronts for 23 glaciers in Greenland. Resulting time series resolve long-term, seasonal, and subseasonal patterns. We discuss the implications of our results and provide the cryosphere community with a data product and an implementation of our processing system.
Weiyan Lin, Jiasong Zhu, Yuansheng Hua, Qingyu Li, Lichao Mou, and Xiao Xiang Zhu
Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XLVIII-1-2024, 371–378, https://doi.org/10.5194/isprs-archives-XLVIII-1-2024-371-2024, https://doi.org/10.5194/isprs-archives-XLVIII-1-2024-371-2024, 2024
Tian Li, Konrad Heidler, Lichao Mou, Ádám Ignéczi, Xiao Xiang Zhu, and Jonathan L. Bamber
Earth Syst. Sci. Data, 16, 919–939, https://doi.org/10.5194/essd-16-919-2024, https://doi.org/10.5194/essd-16-919-2024, 2024
Short summary
Short summary
Our study uses deep learning to produce a new high-resolution calving front dataset for 149 marine-terminating glaciers in Svalbard from 1985 to 2023, containing 124 919 terminus traces. This dataset offers insights into understanding calving mechanisms and can help improve glacier frontal ablation estimates as a component of the integrated mass balance assessment.
Y. Sun, A. Kruspe, L. Meng, Y. Tian, E. J. Hoffmann, S. Auer, and X. X. Zhu
Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XLVIII-1-W2-2023, 225–232, https://doi.org/10.5194/isprs-archives-XLVIII-1-W2-2023-225-2023, https://doi.org/10.5194/isprs-archives-XLVIII-1-W2-2023-225-2023, 2023
J. Zhao, F. Roth, B. Bauer-Marschallinger, W. Wagner, M. Chini, and X. X. Zhu
ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci., X-1-W1-2023, 911–918, https://doi.org/10.5194/isprs-annals-X-1-W1-2023-911-2023, https://doi.org/10.5194/isprs-annals-X-1-W1-2023-911-2023, 2023
M. Mühlhaus, F. Kurz, A. R. Guridi Tartas, R. Bahmanyar, S. M. Azimi, and J. Hellekes
ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci., X-1-W1-2023, 371–378, https://doi.org/10.5194/isprs-annals-X-1-W1-2023-371-2023, https://doi.org/10.5194/isprs-annals-X-1-W1-2023-371-2023, 2023
Yao Sun, Stefan Auer, Liqiu Meng, and Xiao Xiang Zhu
Abstr. Int. Cartogr. Assoc., 6, 250, https://doi.org/10.5194/ica-abs-6-250-2023, https://doi.org/10.5194/ica-abs-6-250-2023, 2023
S. Zhao, S. Saha, and X. X. Zhu
Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XLIII-B3-2022, 1407–1413, https://doi.org/10.5194/isprs-archives-XLIII-B3-2022-1407-2022, https://doi.org/10.5194/isprs-archives-XLIII-B3-2022-1407-2022, 2022
S. Saha, J. Gawlikowski, and X. X. Zhu
Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XLIII-B3-2022, 423–428, https://doi.org/10.5194/isprs-archives-XLIII-B3-2022-423-2022, https://doi.org/10.5194/isprs-archives-XLIII-B3-2022-423-2022, 2022
T. Krauß, F. Kurz, and H. Runge
ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci., V-1-2022, 85–91, https://doi.org/10.5194/isprs-annals-V-1-2022-85-2022, https://doi.org/10.5194/isprs-annals-V-1-2022-85-2022, 2022
K. R. Traoré, A. Camero, and X. X. Zhu
ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci., V-3-2022, 217–224, https://doi.org/10.5194/isprs-annals-V-3-2022-217-2022, https://doi.org/10.5194/isprs-annals-V-3-2022-217-2022, 2022
T. Beker, H. Ansari, S. Montazeri, Q. Song, and X. X. Zhu
ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci., V-3-2022, 85–92, https://doi.org/10.5194/isprs-annals-V-3-2022-85-2022, https://doi.org/10.5194/isprs-annals-V-3-2022-85-2022, 2022
N. Merkle, C. Henry, S. M. Azimi, and F. Kurz
ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci., V-2-2022, 283–289, https://doi.org/10.5194/isprs-annals-V-2-2022-283-2022, https://doi.org/10.5194/isprs-annals-V-2-2022-283-2022, 2022
F. Kurz, P. Mendes, V. Gstaiger, R. Bahmanyar, P. d’Angelo, S. M. Azimi, S. Auer, N. Merkle, C. Henry, D. Rosenbaum, J. Hellekes, H. Runge, F. Toran, and P. Reinartz
ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci., V-1-2022, 221–226, https://doi.org/10.5194/isprs-annals-V-1-2022-221-2022, https://doi.org/10.5194/isprs-annals-V-1-2022-221-2022, 2022
Cited articles
Adrian, J., Sagan, V., and Maimaitijiang, M.: Sentinel SAR-optical fusion for
crop type mapping using deep learning and Google Earth Engine, ISPRS J. Photogramm., 175, 215–235, 2021. a
Al-Najjar, H. A., Kalantar, B., Pradhan, B., Saeidi, V., Halin, A. A., Ueda,
N., and Mansor, S.: Land cover classification from fused DSM and UAV images
using convolutional neural networks, Remote Sensing, 11, 1461, https://doi.org/10.3390/rs11121461, 2019. a
Brachmann, J., Baumgartner, A., and Gege, P.: The Calibration Home Base for
Imaging Spectrometers, Journal of Large-Scale Research Facilities JLSRF, 2, https://doi.org/10.17815/jlsrf-2-137, 2016. a
d'Angelo, P. and Kurz, F.: Aircraft based real time bundle adjustment and
digital surface model generation, in: ISPRS Geospatial Week 2019,
1643–1647, https://elib.dlr.de/127049/ (last access: 2 January 2023), 2019. a
Du, B., Wei, Q., and Liu, R.: An improved quantum-behaved particle swarm
optimization for endmember extraction, IEEE T. Geosci.
Remote, 57, 6003–6017, 2019. a
Filipponi, F.: Sentinel-1 GRD preprocessing workflow, Proceedings, 18, 11, https://doi.org/10.3390/ECRS-3-06201, 2019. a
Ge, C., Du, Q., Sun, W., Wang, K., Li, J., and Li, Y.: Deep Residual
Network-Based Fusion Framework for Hyperspectral and LiDAR Data, IEEE J. Sel. Top. Appl., 14,
2458–2472, 2021. a
Guanter, L., Kaufmann, H., Segl, K., Foerster, S., Rogass, C., Chabrillat, S.,
Kuester, T., Hollstein, A., Rossner, G., Chlebek, C., Straif, C., Fischer,
S., Schrader, S., Storch, T., Heiden, U., Mueller, A., Bachmann, M., Mühle,
H., Müller, R., Habermeyer, M., Ohndorf, A., Hill, J., Buddenbaum, H.,
Hostert, P., Van der Linden, S., Leitão, P. J., Rabe, A., Doerffer, R.,
Krasemann, H., Xi, H., Mauser, W., Hank, T., Locherer, M., Rast, M., Staenz,
K., and Sang, B.: The EnMAP Spaceborne Imaging Spectroscopy Mission for Earth
Observation, Remote Sensing, 7, 8830–8857,
https://doi.org/10.3390/rs70708830, 2015. a
Hang, R., Li, Z., Ghamisi, P., Hong, D., Xia, G., and Liu, Q.: Classification
of hyperspectral and LiDAR data using coupled CNNs, IEEE T.
Geosci. Remote, 58, 4939–4950, 2020. a
Hong, D. and Zhu, X. X.: SULoRA: Subspace unmixing with low-rank attribute
embedding for hyperspectral data analysis, IEEE J. Sel. Top. Signal Process.,
12, 1351–1363, 2018. a
Hong, D., Yokoya, N., Chanussot, J., and Zhu, X.: CoSpace: Common subspace
learning from hyperspectral-multispectral correspondences, IEEE T. Geosci.
Remote, 57, 4349–4359, 2019b. a
Hong, D., Yokoya, N., Ge, N., Chanussot, J., and Zhu, X.: Learnable manifold
alignment (LeMA): A semi-supervised cross-modality learning framework for
land cover and land use classification, ISPRS J. Photogramm. Remote Sens.,
147, 193–205, 2019c. a
Hong, D., Gao, L., Hang, R., Zhang, B., and Chanussot, J.: Deep encoder-decoder
networks for classification of hyperspectral and LiDAR data, IEEE Geosci. Remote S., 19, 5500205, https://doi.org/10.1109/LGRS.2020.3017414, 2020a. a
Hong, D., Wu, X., Ghamisi, P., Chanussot, J., Yokoya, N., and Zhu, X. X.:
Invariant attribute profiles: A spatial-frequency joint feature extractor for
hyperspectral image classification, IEEE T. Geosci. Remote, 58,
3791–3808, 2020b. a
Hong, D., Yokoya, N., Xia, G.-S., Chanussot, J., and Zhu, X. X.: X-ModalNet: A
semi-supervised deep cross-modal network for classification of remote sensing
data, ISPRS J. Photogramm. Remote Sens., 167, 12–23, 2020c. a
Hong, D., Gao, L., Yao, J., Yokoya, N., Chanussot, J., Heiden, U., and Zhang,
B.: Endmember-Guided Unmixing Network (EGU-Net): A General Deep Learning
Framework for Self-Supervised Hyperspectral Unmixing, IEEE Trans. Neural
Netw. Learn. Syst., 33, 6518–6531, https://doi.org/10.1109/TNNLS.2021.3082289, 2021a. a
Hong, D., Gao, L., Yao, J., Zhang, B., Plaza, A., and Chanussot, J.: Graph
convolutional networks for hyperspectral image classification, IEEE Trans.
Geosci. Remote Sens., 59, 5966–5978, 2021b. a
Hong, D., Gao, L., Yokoya, N., Yao, J., Chanussot, J., Qian, D., and Zhang, B.:
More Diverse Means Better: Multimodal Deep Learning Meets Remote-Sensing
Imagery Classification, IEEE T. Geosci. Remote, 59, 4340–4354,
2021c. a
Hong, D., He, W., Yokoya, N., Yao, J., Gao, L., Zhang, L., Chanussot, J., and
Zhu, X.: Interpretable Hyperspectral Artificial Intelligence: When nonconvex
modeling meets hyperspectral remote sensing, IEEE Geosci. Remote Sens. Mag.,
9, 52–87, 2021d. a
Hong, D., Hu, J., Yao, J., Chanussot, J., and Zhu, X. X.: Multimodal remote
sensing benchmark datasets for land cover classification with a shared and
specific feature learning model, ISPRS J. Photogramm., 178, 68–80, 2021e. a
Hong, D., Yao, J., Meng, D., Xu, Z., and Chanussot, J.: Multimodal GANs: Toward
crossmodal hyperspectral-multispectral image segmentation, IEEE T.
Geosci. Remote, 59, 5103–5113, 2021f. a
Hong, D., Yokoya, N., Chanussot, J., Xu, J., and Zhu, X. X.: Joint and
progressive subspace analysis (JPSA) with spatial-spectral manifold alignment
for semisupervised hyperspectral dimensionality reduction, IEEE Trans.
Cybern., 51, 3602–3615, 2021g. a
Hu, J., Ghamisi, P., and Zhu, X. X.: Feature extraction and selection of
sentinel-1 dual-pol data for global-scale local climate zone classification,
ISPRS Int. Geo-Inf., 7, 379, https://doi.org/10.3390/ijgi7090379, 2018. a
Hu, J., Hong, D., and Zhu, X. X.: MIMA: MAPPER-induced manifold alignment for
semi-supervised fusion of optical image and polarimetric SAR data, IEEE
T. Geosci. Remote, 57, 9025–9040, 2019. a
Hu, J., Liu, R., Hong, D., Camero, A., Yao, J., Schneider, M., Kurz, F., Segl,
K., and Zhu, X. X.: MDAS: A new multimodal benchmark dataset for remote
sensing, TUM [data set], https://doi.org/10.14459/2022mp1657312, 2022a. a, b
Hu, J., Liu, R., Hong, D., and Camero, A.: zhu-xlab/augsburg_Multimodal_Data_Set_MDaS: Accepted data set paper, Zenodo [code], https://doi.org/10.5281/zenodo.7428215, 2022b. a, b
Huang, R., Hong, D., Xu, Y., Yao, W., and Stilla, U.: Multi-Scale Local Context
Embedding for LiDAR Point Cloud Classification, IEEE Geosci. Remote S., 17, 721–725, 2020. a
Khodadadzadeh, M., Li, J., Prasad, S., and Plaza, A.: Fusion of hyperspectral
and LiDAR remote sensing data using multiple feature learning, IEEE J.
Sel. Top. Appl., 8,
2971–2983, 2015. a
Köhler, C.: Airborne Imaging Spectrometer HySpex, Journal of Large-Scale
Research Facilities JLSRF, 2, https://doi.org/10.17815/jlsrf-2-151, 2016. a
Krauß, T., d'Angelo, P., Schneider, M., and Gstaiger, V.: The fully automatic optical processing system Catena at DLR, Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XL-1/W1, 177–183, https://doi.org/10.5194/isprsarchives-XL-1-W1-177-2013, 2013. a
Kurz, F., Türmer, S., Meynberg, O., Rosenbaum, D., Runge, H., Reinartz, P.,
and Leitloff, J.: Low-cost Systems for real-time Mapping Applications,
Photogramm. Fernerkun., Schweizerbart Science Publishers, Stuttgart, Germany, 159–176,
https://doi.org/10.1127/1432-8364/2012/0109, 2012. a
Liu, R. and Zhu, X.: Endmember Bundle Extraction Based on Multiobjective
Optimization, IEEE T. Geosci. Remote, 59, 8630–8645, https://doi.org/10.1109/TGRS.2020.3037249, 2020. a
Liu, R., Zhang, L., and Du, B.: A novel endmember extraction method for
hyperspectral imagery based on quantum-behaved particle swarm optimization,
IEEE J. Sel. Top. Appl., 10, 1610–1631, 2017. a
Liu, X., Liu, Q., and Wang, Y.: Remote sensing image fusion based on two-stream
fusion network, Inform. Fusion, 55, 1–15, https://doi.org/10.1016/j.inffus.2019.07.010, 2020. a, b, c, d
Loncan, L., de Almeida, L. B., Bioucas-Dias, J. M., Briottet, X., Chanussot, J., Dobigeon, N., Fabre, S., Liao, W., Licciardi, G. A., Simões, M., Tourneret, J-Y., Veganzones, M. A., Vivone, G., Wei, Q., and Yokoya, N.:
Hyperspectral pansharpening: A review, IEEE Geoscience and remote sensing
magazine, 3, 27–46, https://doi.org/10.1109/MGRS.2015.2440094, 2015. a
Meraner, A., Ebel, P., Zhu, X. X., and Schmitt, M.: Cloud removal in Sentinel-2
imagery using a deep residual neural network and SAR-optical data fusion,
ISPRS J. Photogramm., 166, 333–346, 2020. a
Okujeni, A., van der Linden, S., and Hostert, P.: Berlin-urban-gradient dataset
2009 – an enmap preparatory flight campaign, EnMAP Flight Campaigns Technical Report, Potsdam: GFZ Data Services, https://doi.org/10.2312/enmap.2016.002, 2016. a
Paris, C. and Bruzzone, L.: A three-dimensional model-based approach to the
estimation of the tree top height by fusing low-density LiDAR data and very
high resolution optical images, IEEE T. Geosci. Remote, 53, 467–480, 2014. a
Rainforth, T. and Wood, F.: Canonical correlation forests, arXiv [preprint],
https://doi.org/10.48550/arXiv.1507.05444, 20 July 2015. a
Rasti, B., Hong, D., Hang, R., Ghamisi, P., Kang, X., Chanussot, J., and
Benediktsson, J.: Feature Extraction for Hyperspectral Imagery: The Evolution
from Shallow to Deep: Overview and Toolbox, IEEE Geosci. Remote Sens. Mag.,
8, 60–88, 2020. a
Richter, R.: Correction of satellite imagery over mountainous terrain, Appl.
Opt., 37, 4004–4015, 1998. a
Rottensteiner, F., Sohn, G., Jung, J., Gerke, M., Baillard, C., Benitez, S., and Breitkopf, U.: The ISPRS benchmark on urban object classification and 3D building reconstruction, ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci., I-3, 293–298, https://doi.org/10.5194/isprsannals-I-3-293-2012, 2012. a
Schläpfer, D., Richter, R., and Feingersh, T.: Operational BRDF effects
correction for wide-field-of-view optical scanners (BREFCOR), IEEE
T. Geosci. Remote, 53, 1855–1864, 2014. a
Schwind, P., Schneider, M., and Müller, R.: Improving HySpex Sensor Co-registration Accuracy using BRISK and Sensor-model based RANSAC, Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XL-1, 371–376, https://doi.org/10.5194/isprsarchives-XL-1-371-2014, 2014. a
Segl, K., Guanter, L., Kaufmann, H., Schubert, J., Kaiser, S., Sang, B., and
Hofer, S.: Simulation of Spatial Sensor Characteristics in the Context of the
EnMAP Hyperspectral Mission, IEEE T. Geosci. Remote, 48, 3046–3054, https://doi.org/10.1109/TGRS.2010.2042455, 2010. a, b
Segl, K., Guanter, L., Rogass, C., Kuester, T., Roessner, S., Kaufmann, H.,
Sang, B., Mogulsky, V., and Hofer, S.: EeteS – The EnMAP End-to-End
Simulation Tool, IEEE J. Sel. Top. Appl., 5, 522–530,
https://doi.org/10.1109/JSTARS.2012.2188994, 2012. a, b
Segl, K., Guanter, L., Gascon, F., Kuester, T., Rogass, C., and Mielke, C.:
S2eteS: An End-to-End Modeling Tool for the Simulation of Sentinel-2 Image
Products, IEEE T. Geosci. Remote, 53, 5560–5571,
https://doi.org/10.1109/TGRS.2015.2424992, 2015. a, b
Sheikholeslami, M. M., Nadi, S., Naeini, A. A., and Ghamisi, P.: An efficient
deep unsupervised superresolution model for remote sensing images, IEEE
J. Sel. Top. Appl.,
13, 1937–1945, 2020. a
Sumbul, G., Charfuelan, M., Demir, B., and Markl, V.: Bigearthnet: A
large-scale benchmark archive for remote sensing image understanding, in:
IGARSS 2019–2019 IEEE International Geoscience and Remote Sensing Symposium, Yokohama, Japan, 28 July–2 August 2019,
5901–5904, https://doi.org/10.1109/IGARSS.2019.8900532, 2019. a
Tupin, F. and Roux, M.: Detection of building outlines based on the fusion of
SAR and optical features, ISPRS J. Photogramm.,
58, 71–82, 2003. a
Wu, X., Hong, D., Tian, J., Chanussot, J., Li, W., and Tao, R.: ORSIm detector:
A novel object detection framework in optical remote sensing imagery using
spatial-frequency channel features, IEEE T. Geosci. Remote, 57,
5146–5158, 2019. a
Wu, X., Hong, D., Chanussot, J., Xu, Y., Tao, R., and Wang, Y.: Fourier-based
Rotation-invariant Feature Boosting: An Efficient Framework for Geospatial
Object Detection, IEEE Geosci. Remote S., 17, 302–306, 2020. a
Xia, G.-S., Yang, W., Delon, J., Gousseau, Y., Sun, H., and Maître, H.:
Structural high-resolution satellite image indexing, in: ISPRS TC VII
Symposium – 100 Years ISPRS, vol. 38, 298–303, 2010. a
Xia, G.-S., Hu, J., Hu, F., Shi, B., Bai, X., Zhong, Y., Zhang, L., and Lu, X.:
AID: A benchmark data set for performance evaluation of aerial scene
classification, IEEE T. Geosci. Remote, 55,
3965–3981, 2017. a
Xia, G.-S., Bai, X., Ding, J., Zhu, Z., Belongie, S., Luo, J., Datcu, M.,
Pelillo, M., and Zhang, L.: DOTA: A large-scale dataset for object detection
in aerial images, in: Proceedings of the IEEE conference on computer vision
and pattern recognition, Salt Lake City, Utah, US, 18–22 June 2018, 3974–3983, 2018. a
Xu, Y., Du, B., Zhang, L., Cerra, D., Pato, M., Carmona, E., Prasad, S.,
Yokoya, N., Hänsch, R., and Le Saux, B.: Advanced multi-sensor optical
remote sensing for urban land use and land cover classification: Outcome of
the 2018 IEEE GRSS data fusion contest, IEEE J. Sel. Top.
Appl., 12, 1709–1724, 2019. a
Yang, Y. and Newsam, S.: Bag-of-visual-words and spatial extensions for
land-use classification, in: Proceedings of the 18th SIGSPATIAL international
conference on advances in geographic information systems, San Jose, California, US, 2–5 November 2010, 270–279, https://doi.org/10.1145/1869790.1869829, 2010. a
Yao, J., Hong, D., Xu, L., Meng, D., Chanussot, J., and Xu, Z.:
Sparsity-Enhanced Convolutional Decomposition: A Novel Tensor-Based Paradigm
for Blind Hyperspectral Unmixing, IEEE T. Geosci. Remote, 60, 5505014, https://doi.org/10.1109/TGRS.2021.3069845, 2021. a
Zhang, D., Shao, J., Li, X., and Shen, H. T.: Remote sensing image
super-resolution via mixed high-order attention network, IEEE T.
Geosci. Remote, 59, 5183–5196, 2020. a
Zhang, S., Yuan, Q., Li, J., Sun, J., and Zhang, X.: Scene-adaptive remote
sensing image super-resolution using a multiscale attention network, IEEE
T. Geosci. Remote, 58, 4764–4779,
2020. a
Zhang, T., Zhang, X., Li, J., Xu, X., Wang, B., Zhan, X., Xu, Y., Ke, X., Zeng, T., Su, H., Ahmad, I., Pan, D., Liu, C., Zhou, Y., Shi., J., and Wei, S.: SAR Ship Detection Dataset (SSDD): Official Release and
Comprehensive Data Analysis, Remote Sensing, 13, 3690, https://doi.org/10.3390/rs13183690, 2021. a
Zhou, Y., Wetherley, E. B., and Gader, P. D.: Unmixing urban hyperspectral
imagery using probability distributions to represent endmember variability,
Remote Sens. Environ., 246, 111857, https://doi.org/10.1016/j.rse.2020.111857, 2020. a
Zhu, F.: Hyperspectral unmixing: ground truth labeling, datasets, benchmark
performances and survey, arXiv [preprint], https://doi.org/10.48550/arXiv.1708.05125, 17 August 2017. a
Zhu, X. X., Hu, J., Qiu, C., Shi, Y., Kang, J., Mou, L., Bagheri, H., Haberle, M., Hua, Y., Huang, R., Hughes, L., Li, H., Sun, Y., Zhang, G., Han, S., Schmitt, M., and Wang, Y.: So2Sat LCZ42: A Benchmark Data Set for the Classification of Global Local Climate Zones [software and data set], IEEE Geoscience and Remote Sensing Magazine, 8, 76–89, https://doi.org/10.1109/MGRS.2020.2964708, 2020. a, b
Zhuang, L., Lin, C.-H., Figueiredo, M. A., and Bioucas-Dias, J. M.:
Regularization parameter selection in minimum volume hyperspectral unmixing,
IEEE T. Geosci. Remote, 57, 9858–9877, 2019. a
Short summary
Multimodal data fusion is an intuitive strategy to break the limitation of individual data in Earth observation. Here, we present a multimodal data set, named MDAS, consisting of synthetic aperture radar (SAR), multispectral, hyperspectral, digital surface model (DSM), and geographic information system (GIS) data for the city of Augsburg, Germany, along with baseline models for resolution enhancement, spectral unmixing, and land cover classification, three typical remote sensing applications.
Multimodal data fusion is an intuitive strategy to break the limitation of individual data in...
Altmetrics
Final-revised paper
Preprint