SoilKsatDB: global database of soil saturated hydraulic conductivity measurements for geoscience applications

The saturated soil hydraulic conductivity (K_sat) is a key parameter in many hydrological and climate models. K_sat values are primarily determined from basic soil properties and may vary over several orders of magnitude. Despite the availability of K_sat datasets in the literature, significant efforts are required to combine the data before they can be used for specific applications. In this work, a total of 13 258 K_sat measurements from 1908 sites were assembled from the published literature and other sources, standardized (i.e., units made identical), and quality checked in order to obtain a global database of soil saturated hydraulic conductivity (SoilKsatDB). The SoilKsatDB covers most regions across the globe, with the highest number of K_sat measurements from North America, followed by Europe, Asia, South America, Africa, and Australia. In addition to K_sat, other soil variables such as soil texture (11 584 measurements), bulk density (11 262 measurements), soil organic carbon (9787 measurements), moisture content at field capacity (7382), and wilting point (7411) are also included in the dataset. To show an application of SoilKsatDB, we derived K_sat pedotransfer functions (PTFs) for temperate regions and laboratory-based soil properties (sand and clay content, bulk density). Accurate models can be fitted using a random forest machine learning algorithm (best concordance correlation coefficient (CCC) equal to 0.74 and 0.72 for temperate area and laboratory measurements, respectively). However, when these K_sat PTFs are applied to soil samples obtained from tropical climates and field measurements, respectively, the model performance is significantly lower (CCC = 0.49 for tropical and CCC = 0.10 for field measurements). These results indicate that there are significant differences between K_sat data collected in temperate and tropical regions and K_sat measured in the laboratory or field. The SoilKsatDB dataset is available at https://doi.org/10.5281/zenodo.3752721 (Gupta et al., 2020) and the code used to extract the data from the literature and the applied random forest machine learning approach are publicly available under an open data license.

1 Introduction

The soil saturated hydraulic conductivity (K_sat) describes the rate of water movement through saturated soils and is defined as the ratio between water flux and hydraulic gradient (Amoozegar and Warrick, 1986). It is a key variable in a number of hydrological, geomorphological, and climatological applications, such as rainfall partitioning into infiltration and runoff (Vereecken et al., 2010), optimal irrigation design (Hu et al., 2015), and the prediction of natural hazards including catastrophic floods and landslides (Batjes, 1996; Gliński et al., 2000; Zhang et al., 2018). Accurate measurements of K_sat in the laboratory and field are laborious and time consuming and are often scale dependent (Youngs, 1991). Using infiltrometer measurements in the field also enables the measurement of K_sat in forests and other types of structured soils; however, so far K_sat values have been measured mainly for agricultural soils (Romano and Palladino, 2002).

Efforts to produce reliable and spatially refined datasets of hydraulic properties date back to the 1970s with the proliferation of distributed hydrologic and climatic modeling. These early notable works also provided basic databases (some of which are used in this study) for Australia (McKenzie et al., 2008; Forrest et al., 1985), Belgium (Vereecken et al., 2017; Cornelis et al., 2001), Brazil (Tomasella et al., 2000, 2003; Ottoni et al., 2018), France (Bruand et al., 2004), Germany (Horn et al., 1991; Krahmer et al., 1995), Hungary (Nemes, 2002), the Netherlands (Wösten et al., 2001), Poland (Glinski et al., 1991), and the USA (Rawls et al., 1982). A detailed discussion of the available datasets for K_sat and other hydrophysical properties is provided in Nemes (2011). Collaborative efforts have resulted in the compilation of multiple databases, including the Unsaturated Soil Hydraulic Database (UNSODA) (Nemes et al., 2001), the Grenoble Catalogue of Soils (GRIZZLY) (Haverkamp et al., 1998), and the Mualem catalogue (Mualem, 1976). These databases, however, focused on soil types and not on the spatial context of K_sat mapping. In an effort to provide spatial context, Jarvis et al. (2013) and Rahmati et al. (2018) published global databases for soil hydraulic and soil physical properties. Likewise, the European soil data center also started projects, such as SPADE (Hiederer et al., 2006) and HYPRES (Wösten, 2000), for generating spatially referenced soil databases for several countries. Since HYPRES only includes western European countries, Weynants et al. (2013) gathered data from 18 countries and developed the European HYdropedological Data Inventory (EU-HYDI) database. This dataset is, however, not publicly available and was not included in this compilation. The datasets mentioned above cover almost all climatic zones except tropical regions, where K_sat values can be significantly different due to the strong local weathering processes and different clay mineralogy (Hodnett and Tomasella, 2002). Recently, Ottoni et al. (2018) published a dataset named HYBRAS (Hydrophysical Database for Brazilian Soils) improving the coverage of South American tropical regions. In addition, Rahmati et al. (2018) recently published the Soil Water Infiltration Global database (SWIG) with information on K_sat for the whole globe. In the SWIG database, some K_sat values were extracted from the literature and other K_sat values were deduced from infiltration time series. In contrast to laboratory measurements that determine K_sat as the ratio of flux density to gradient, infiltration-based methods determine K_sat by fitting infiltration dynamics to parametric models of the infiltration process; for a review on analytical models characterizing the infiltration process, see Kutílek et al. (1988), Youngs (1991), and Vereecken et al. (2019).

The increasing demand for highly resolved descriptions of surface processes requires commensurate advances in the representation of K_sat in modern Earth system model (ESM) applications. Several existing K_sat datasets are either missing coordinates or these are provided with an unknown accuracy, thus limiting their applications for spatial modeling. For example, the SWIG dataset is missing information on soil depth and assigns entire watersheds to a single coordinate. Similarly, the UNSODA dataset does not provide coordinates and soil texture information for all samples. For a few locations, HYBRAS uses a different coordinate system. Taken together, these limitations imply that to prepare spatially referenced global K_sat datasets for large scale applications, it requires serious effort to compile, standardize, and quality check all (publicly available) literature.

The objective of the work here is to provide a new global standardized K_sat database (SoilKsatDB) that can be used for geoscience applications. To do so, a total of 13 258 K_sat measurements was compiled, standardized, and crosschecked to produce a harmonized compilation that is analysis-ready (i.e., it can directly be used to test various machine learning algorithms for spatial analysis). We compiled data from existing datasets and, to improve the spatial coverage in regions with sparse data, we conducted a literature search to include K_sat measurements in geographic areas that were not yet included in other existing databases. In this paper, we first describe the data compilation process and then describe methodological steps used to spatially reference, filter, and standardize the existing datasets. As an illustrative application of the dataset, we derive pedotransfer functions (PTFs) for different climatic regions and measurement methods and discuss their transferability to other regions and measurement methodologies. We fully document all importing, standardization, and binding steps using the R environment for statistical computing (R Core Team, 2013), so that we can collect feedback from other researchers and increase the speed of further updates and improvements. The newly created dataset (SoilKsatDB) can be accessed via https://doi.org/10.5281/zenodo.3752721 (Gupta et al., 2020).

2 Methods and materials

2.1 Data sources

To locate and obtain all compatible datasets, a literature search was conducted using different search engines, including Science Direct (https://www.sciencedirect.com/, last access: 9 February 2021), Google Scholar (https://scholar.google.com/, last access: 9 February 2021), and Scopus (https://www.scopus.com, last access: 11 January 2021). We searched soil hydraulic conductivity datasets using “saturated hydraulic conductivity database”, “Ksat”, and “hydraulic conductivity curves” as keywords. The collected datasets are listed in Table 1 together with the number of K_sat observations for each study. They can be classified into three main categories, namely: (i) existing datasets (in the form of tables) published and archived with a DOI in peer-reviewed publications, (ii) legacy datasets in paper or document format (e.g., legacy reports, PhD theses, and scientific studies), and (iii) online materials.

Existing datasets include published datasets such as HYBRAS (Ottoni et al., 2018), UNSODA (Nemes et al., 2001), SWIG (Rahmati et al., 2018), and the soil hydraulic properties over the Tibetan Plateau (Zhao et al., 2018), from which we extracted the required information as described in Table 2. The major challenge with making the existing datasets compatible for binding (standardization, removing redundancy) was to obtain the locations for a particular sample as well as the corresponding measurement depths. For instance, the UNSODA database does not provide information on the geographical locations. To fill the gaps and make the data suitable also for spatial analysis, we used Google Earth to find the coordinates based on the given location (generally an address or a location name). Moreover, all datasets were crosschecked to avoid redundancy. For example, the UNSODA data include the data of Vereecken et al. (2017) and Richard and Lüscher (1983/87) while the SWIG database includes the measurements of Zhao et al. (2018). Hence we removed these from the UNSODA and SWIG database and used the original sources.

Rycroft et al. (1975)Habel (2013)Wang et al. (2008)Waddington and Roulet (1997)Nyman et al. (2011)Deshmukh et al. (2014)Takahashi (1997)Bhattacharyya et al. (2006)Price et al. (2010)Katimon and Hassan (1997)Lopes et al. (2020)Bonsu and Masopeh (1996)El-Shafei et al. (1994)Yasin and Yulnafatmawita (2018)Bambra (2016)Lopez et al. (2015)Daniel et al. (2017)Verburg et al. (2001)Kramarenko et al. (2019)Arend (1941)Southard and Buol (1988)Zakaria (1992)Helbig et al. (2013)Chang (2010)Ramli (1999)Gwenzi et al. (2011)Yao et al. (2013)Singh et al. (2011)Päivänen (1973)Becker et al. (2018)Campbell et al. (1977)Mahapatra and Jha (2019)Baird et al. (2017)Chief et al. (2008)Amer et al. (2009)Keisling (1974)Conedera et al. (2003)Radcliffe et al. (1990)Rahimy (2011)Ebel et al. (2012)Vogeler et al. (2019)Hao et al. (2019)Ferreira et al. (2005)Singh et al. (2006)Kanemasu (1994)Imeson et al. (1992)Kelly et al. (2014)Tete-Mensah (1993)Johansen et al. (2001)Elnaggar (2017)Zhao et al. (2018)Lamara and Derriche (2008)Ganiyu et al. (2018)Hinton (2016)Parks and Cundy (1989)Cisneros et al. (1999)Vieira and Fernandes (2004)Ravi et al. (2017)Niemeyer et al. (2014)Houghton (2011)Smettem and Ross (1992)Sharratt (1990)Tian et al. (2017)Boike et al. (1998)Habecker et al. (1990)Li et al. (2017)Andrade (1971)Nielsen et al. (1973)Forrest et al. (1985)Beyer et al. (2015)Robbins (1977)Richard and Lüscher (1983/87)Blake et al. (2010)Sonneveld et al. (2005)Sanzeni et al. (2013)Bonell and Williams (1986)Quinton et al. (2008)Vereecken et al. (2017)Kutiel et al. (1995)Simmons (2014)Coelho (1974)Martin and Moody (2001)Ouattara (1977)Kool et al. (1986)Mott et al. (1979)Hardie et al. (2011)Nemes et al. (2001)Rab (1996)Baird (1997)Ottoni et al. (2018)Soracco et al. (2010)Kirby et al. (2001)Rahmati et al. (2018)Varela et al. (2015)Yoon (2009)Grunwald (2020)Sayok et al. (2007)Jabro (1992)Abagandura et al. (2017)Greenwood and Buttle (2014)

Table 1 List of articles, digitized K_sat datasets, and the number of points (N) per dataset used to generate the SoilKsatDB.

In the case of legacy datasets (non-digital tabular format, non-peer-reviewed data), we invested significant effort to digitize, clean, and crosscheck the data to extract K_sat values. Two datasets were also collected directly from project websites providing data on hydraulic and thermal conductivity (retrieved from https://daac.ornl.gov/FIFE/guides/Soil_Hydraulic_Conductivity_Data.html, last access: 8 January 2021, and described in Kanemasu (1994)) and the Florida database (http://soils.ifas.ufl.edu, last access: 8 January 2021) from Grunwald (2020).

There are many biomes and climatic regions, such as desert dunes, peatlands, and frozen soils, for which very few K_sat measurements were publicly available. We have intensively searched for additional data for these areas and found 39 studies (each with less than 5 K_sat measurements) to cover these regions. We thus digitized K_sat values from these studies (shown either in bar charts or line plots), georeferenced the maps where necessary, and then converted the data into tabular form. In some cases, we also contacted colleagues that worked in these regions to retrieve additional data.

Figure 1 Spatial distribution of K_sat measurements based on (red) laboratory and (blue) field measurements in the SoilKsatDB. A total of 1908 locations are shown on the map.

2.2 Georeferencing K_sat values and definition of spatial accuracy

Georeferencing of K_sat measurements is important for using the data for local, regional, or global hydrological and land surface models. Although many studies provided information on the geographical location of the measurements, studies conducted particularly in the 1970s and 1980s only provided the name of the locations and approximate distance from a reference location. A limited accuracy of the position value may affect the application of the K_sat value in a spatially distributed model. For example, in the case of a location with contrasting hydraulic properties, it must be known to which subregion the measured value can be assigned and the user must know if the given location is accurate enough. For that purpose we assigned an accuracy value (“accuracy classes”) to each measurement as explained next. We assigned each K_sat value to one of seven “accuracy classes” ranging from highest (0–100 m) to lowest (more than 10 000 m or non-available information (NA)) accuracy. For example, Forrest et al. (1985), Zhao et al. (2018), and Ottoni et al. (2018) provided exact coordinates of the locations, thus we assigned a location accuracy of 0–100 m (i.e., highly accurate; see Table 3 for more details). For other references, we digitized provided maps or sketches with locations of the points. We first georeferenced these maps using ESRI ArcGIS software (v10.3) and then digitized the coordinates from georeferenced images. Some of the documents we digitized (e.g., Nemes et al., 2001) provided the names of specific locations and hence we used Google Earth to obtain the coordinates. We estimate that the spatial location accuracy of these points is roughly between 0 and 5 km. Similarly, spatial maps in jpg format (e.g., Becker et al., 2018) were georeferenced with 100–500 m location accuracy. In contrast, few studies (e.g., Yoon, 2009) provided the exact location of the sampling with assumed location accuracy of 10–20 m. In the SWIG database, the information related to location (coordinates for each point) was missing, so we went through each publication referenced in Rahmati et al. (2018) and added coordinates.

Table 2 (a) Description and units of some key variables listed in the database. The list can be found in the readme file (Table 1) and dataset “sol_ksat.pnts_horizons” on Zenodo (https://doi.org.10.5281/zenodo.3752721). We used the same codes as in the National Cooperative Soil Survey (NCSS) Soil Characterization Database (National Cooperative Soil Survey, 2016). (b) Example of K_sat database structure with key variables (from left to right: unique ID, reference, longitude and latitude (decimal degree), minimum and maximum accuracy (m), top and bottom of soil sample (cm), horizon designation, bulk density (g cm⁻³), moisture content at field capacity and wilting point (%), soil textural class, clay, silt and sand content (%), soil organic carbon content (%), soil acidity, saturated hydraulic conductivity measured in lab or field (cm d⁻¹), source of the data, location id, and mean soil depth). NA is “no value”. Column names are explained in (a). The dataset file “sol_ksat.pnts_horizons” can be found on Zenodo (https://doi.org.10.5281/zenodo.3752721).

Table 3 Number of samples (N) assigned to each spatial accuracy class. NA are samples without information on spatial accuracy.

(Klute and Dirksen, 1986)(Bagarello and Sgroi, 2004)(Klute, 1965)(Bodhinayake et al., 2004)(Purdy and Suryasasmita, 2006)(Lassabatere et al., 2006)(Sharratt, 1990)(Naik et al., 2019)(Sutejo et al., 2019)(Reynolds et al., 2000)(Soracco et al., 2010)(Terzaghi, 2004)(Gupta et al., 1993)(Schwärzel and Punzel, 2007)(Sepehrnia et al., 2016)(Reynolds and Elrick, 1985)(Hinton, 2016)(Mohanty et al., 1994)(Amoozegar, 1989)(Muñoz-Carpena et al., 2002)(Baird et al., 2017)(Nielsen et al., 1973)(Gupta et al., 1993)(Scotter et al., 1982)(Mohsenipour and Shahid, 2016)

Table 4 Instruments and methods used to estimate K_sat. A key reference with further details is given for all methods. The field methods are subdivided into different subcategories (in some cases, “ponding” in the field methods or “permeameter” in the lab methods were listed in original studies without specification). Information on the applied method for each sample is provided in file “sol_ksat.pnts_metadata_cl_pedo.csv” (https://doi.org/10.5281/zenodo.3752721).

2.3 Standardization

The database was cleaned to remove 716 unrealistic low K_sat values as outliers deduced from infiltration time series in the SWIG database. Moreover, in the SWIG database, soil depth information was not available, so we assumed that infiltration experiments were conducted in the topsoil and assigned a depth of 0–20 cm. Furthermore, we computed sand (particles > 50 µm), silt (2–50 µm), and clay fraction (<2 µm) for the UNSODA database based on the available particle-size data, assuming a log-normal distribution, as described in Nemes et al. (2001).

After data extraction from the literature, georeferencing, and standardization (conversion of all values to the same units), all information was collected in tabulated form in the new database SoilKsatDB (https://doi.org/10.5281/zenodo.3752721). The database consists of 23 columns (various sample properties) and 13 259 rows (a header and 13 258 samples). An excerpt of the database with all 23 columns is shown in Table .

Table 5 Mean values of soil hydrophysical properties for each soil textural class. The number of samples (N) is given in parentheses under each soil variable for each soil texture classes. N values marked with ^* correspond to undefined soil texture class. BD: bulk density (g cm⁻³); OC: soil organic carbon content (%); FC: moisture content at field capacity (vol %); WP: moisture content at wilting point (vol %); $K_{{\mathrm{sat}}_{\mathrm{l}}}$ and $K_{{\mathrm{sat}}_{\mathrm{f}}}$ are laboratory and field K_sat (cm d⁻¹), respectively. For K_sat, the geometric mean is reported (due to the sensitivity to a few extreme values for the arithmetic mean). For all other properties the arithmetic mean is provided.

2.4 Statistical modeling of K_sat

To show a possible application of the database, we computed various PTFs. The PTF models were fitted using a random forest (RF) machine learning algorithm (Breiman, 2001) in the R environment for statistical computing (R Core Team, 2013). We fitted the RF model for log-transformed (log₁₀) K_sat values as a function of primary soil properties. In this application, PTFs for K_sat were built based on bulk density and sand and clay content. The observed correlation between these primary soil properties and K_sat motivated us to use key variables for the estimation of PTFs. Organic carbon (OC) was not used to build the PTFs because (i) this information was missing for 15 % of measurements and (ii) the correlation between OC and K_sat was poor (i.e., 0.005; Pearson correlation coefficient). We derived two PTFs for K_sat:

1. PTF for temperate regions. The map of K_sat locations were overlaid on the Köppen–Geiger climate zone map (Rubel and Kottek, 2010; Hamel et al., 2017) and then divided the measurements based on climatic regions (temperate, tropical, boreal, and arid) to account for differences in climate and related weathering processes (Hodnett and Tomasella, 2002). A total of 8296 K_sat values for measurements in temperate climate that contain information on sand, clay, and bulk density, were used to develop the PTF. The dataset was randomly divided into a training (6637 samples, 80 %) and testing dataset (1659 samples, 20 %). PTFs for temperate regions include all depths of K_sat measurements (40 % K_sat values from the top 0-20 cm of soil). In a validation step, we applied the PTF determined for temperate regions to tropical regions. Compared to temperate regions, the tropics are affected by different soil formation processes resulting in different clay mineral types. With such validation, we intend to discuss the transferability of PTFs across different regions.

2. PTF for laboratory-based K_sat values. In a second application, the dataset (total 13 258) was divided into laboratory- and field-based K_sat values. A total of 9155 K_sat measurements belong to laboratory and 4131 K_sat measurements belong to field values. The laboratory dataset (8491 K_sat measurements with information on soil texture and bulk density information) was used for training (6793) and testing (1698) following the same method as used for the PTF for the temperate climate (i.e., 80 % for training and 20 % for testing). Lab-based PTFs includes all depths of K_sat measurements (30 % K_sat values from the top 0-20 cm of soil). Similar to the application of the PTF from the temperate region for the tropics, we apply the PTF deduced from laboratory data for prediction of K_sat measured in the field. We expect differences because field measurements scan larger soil volumes that may contain soil structural pores.

The “ranger” package version 0.12.1 (Wright and Ziegler, 2015) was used to build the PTFs. The PTFs developed for temperate regions and for laboratory data were then applied to test their ability to predict the result for the measurements in the tropical climate (1111 K_sat measurements) and for field measurements (1998 K_sat measurements with information on soil texture and bulk density information), respectively. The code for generating and testing the PTFs is provided in the Supplement. ANOVA (analysis of variance) with post hoc Tukey's HSD (honestly significant difference) test (Hilton and Armstrong, 2006; Abdi and Williams, 2010) was used to test the significant difference in K_sat between texture classes. The ANOVA test indicates that at least one group differs from the other groups but does not explain the patterns of differences between means. Then, the Tukey HSD test was used to compute the significant difference between two means using a statistical distribution as shown in the results in Appendix A. This analysis was important to understand and check whether K_sat values between soil texture classes are significantly different or not.

The relative importance of the covariates for modeling K_sat was assessed by the node impurity, which for RF regression problems is computed as the decrease of residual sum of squares (RSS) when a particular covariate splits the data at the nodes of a tree (Hastie et al., 2009, Sects. 10.13.1, 15.3.2). The variable that provides maximum decline in RSS (and consequently increase in node purity) is considered the most important variable; the variable with the second largest RSS decrease is considered the second most important variable, and so on. Furthermore, the accuracy of the predictions was evaluated using bias, root mean square error (RMSE, in log-transformed K_sat measurement), and concordance correlation coefficient (CCC) (Lawrence and Lin, 1989).

Bias and RMSE are defined as

$$\begin{array}{}\text{(1)}& {\displaystyle }\text{bias}=\sum _{i=\mathrm{1}}^n{\displaystyle \frac{({\widehat{y}}_i-y_i)}{n}}\text{(2)}& {\displaystyle }\text{RMSE}=\sqrt{\sum _{i=\mathrm{1}}^n{\displaystyle \frac{({\widehat{y}}_i-y_i{)}^{\mathrm{2}}}{n}}},\end{array}$$

where y and $\widehat{y}$ are observed and predicted K_sat values, respectively, and n is the total number of cross-validation points.

The CCC is a measure of the agreement between observed and predicted K_sat value and is computed as

$$\begin{array}{}\text{(3)}& \text{CCC}={\displaystyle \frac{\mathrm{2}\cdot \mathit{\rho }\cdot {\mathit{\sigma }}_{\widehat{y}}\cdot {\mathit{\sigma }}_y}{{\mathit{\sigma }}_{\widehat{y}}^{\mathrm{2}}+{\mathit{\sigma }}_y^{\mathrm{2}}+({\mathit{\mu }}_{\widehat{y}}-{\mathit{\mu }}_y{)}^{\mathrm{2}}}},\end{array}$$

where ${\mathit{\mu }}_{\widehat{y}}$ and μ_y are predicted and observed means, ${\mathit{\sigma }}_{\widehat{y}}$ and σ_y are predicted and observed variances, and ρ is the Pearson correlation coefficient between predicted and observed values. CCC is equal to 1 for a perfect model.

3 Results

3.1 Data coverage of SoilKsatDB

Based on the literature search and data compilation, we have assembled a total of 13 258 values of K_sat from 1908 locations (each location has a unique location_id). Moreover, the database contains a total of 13 286 K_sat values because a few studies have reported both field and lab measurements for the same location. Figure 1 shows the global distribution of the sites used in this study. Most data originate from North America, followed by Europe, Asia, South America, Africa, and Australia. With respect to climatic regions, 10 093 K_sat measurements were taken in temperate regions (8296 contained texture and bulk density information and were used to build PTFs) and 1443, 1106, 580, and 36 in tropical, arid, boreal, and polar regions, respectively, as shown in Fig. 2b. The points are often spatially clustered with the biggest cluster of points (1103 locations with 6532 K_sat measurements) in Florida (Grunwald, 2020). The K_sat database includes 4131 values from field measurement and 9155 values from laboratory measurements. In particular, different types of infiltrometers (e.g., Mini disc infiltrometer, Tension infiltrometer) and permeameters (e.g., Guelph permeameter, Aardvark permeameter) were used for the field measurements, whereas constant or falling head methods were mainly used in laboratory analyses (Table 4).

Out of the 13 258 K_sat measurements, 11 584 had information on soil texture, 11 262 on bulk density, 9787 on organic carbon, 7382 on field capacity, and 7411 on wilting point, while for 8994 measurements information for all basic soil properties (bulk density, soil texture, and organic carbon) was available (Fig. 2a).

The methods used to compute these soil properties (as much as we could extract from the literature and existing databases) are listed in the CSV file sol_ksat.pnts_metadata_cl_pedo.csv available at https://doi.org/10.5281/zenodo.3752721. Note that in addition to 11 584 soil texture values, 75 measurements have soil texture information with total (sand + silt + clay) less than 98 % or greater than 102 %. We did not use these values in the PTF development but included it in the database as “Error” class in the soil texture column.

Figure 2 Description of K_sat measurements. (a) Venn diagram illustrating the number of K_sat measurements in the SoilKsatDB for which bulk density, soil texture, and soil organic carbon data were also available. Note that the size of the intersecting areas does not represent the correct fractions. (b) Distribution of K_sat measurements among climatic regions.

3.2 Statistical characteristics of SoilKsatDB

The distribution of measurements based on soil texture classes is shown on the USDA soil texture triangle in Fig. 3a. The database covers all textural classes, with a high clustering in sandy soils due to the numerous samples from Florida (Grunwald, 2020), while only few measurements belong to the silt textural class. The increase in K_sat values in clayey and loamy soils for field methods (compared to laboratory methods) is likely due to the effect of soil structure. ANOVA with post hoc Tukey's HSD test showed that the mean values for all broad soil texture classes are significantly different from each other, except for clayey soils field K_sat values and sandy soils field K_sat values (see Table A1). The violin distribution plot in Fig. 3c shows the range of K_sat values for the different databases. Most of the datasets report K_sat values between 10⁻² and 10^2.5 cm d⁻¹, with a wider range of K_sat values observed in measurements from theses and reports (including studies with extreme values from sandy desert soils and low conductive clay soils) and from the SWIG database (databases 9 and 6 in Fig. 3c, respectively). Likewise, Fig. 3d shows the violin distribution of K_sat based on soil texture classes. The arithmetic mean of K_sat was highest for the sand and loamy sand soils (i.e., 2.68 and 1.99, respectively, in log ₁₀ cm d⁻¹), while the lowest mean values were found for silt and silty loam (i.e., 1.12 and 1.15, respectively, in log ₁₀ cm d⁻¹). Table A2 shows that the K_sat values in sand and loamy sand soil texture classes are significantly different from all other soil texture classes. However, silt, silty clay, and silty clay loam classes are not significantly different from clay, sandy clay, and sandy clay loam K_sat values.

Figure 3 Characterization of collected K_sat values: (a) distribution of soil samples on the USDA soil texture triangle, (b) distribution of K_sat values using broad soil texture classes (sandy soils: sand and loamy sand; loamy soils: sandy loam, loam, silt loam, silt, clay loam, and sandy clay loam; clayey soils: sandy clay, silty clay, and clay) based on laboratory and field measurements (the number of measurements is shown on the top of the figure). Panel (c) shows the range of K_sat values spanned by each data source. The database numbers 1–9 refer to different sources and databases. 1: Australia (Forrest et al., 1985), 2: Belgium (Vereecken et al., 2017), 3: China (Tian et al., 2017; Li et al., 2017), 4: Florida (Grunwald, 2020), 5: HYBRAS (Ottoni et al., 2018), 6: SWIG (Rahmati et al., 2018), 7: Tibetan Plateau (Zhao et al., 2018), 8: UNSODA (Nemes et al., 2001), 9: all other databases in Table 1. (d) Distribution of K_sat based on soil textural classes with the number of measurements shown on the top of the figure. In the violin diagrams (c, d), the dot represents the mean value and the line represents the standard deviation for each dataset.

Average values of K_sat and other hydrophysical properties are shown in Table 5. Higher average organic carbon and bulk density values were observed in clayey and loamy soils compared to sandy soils. K_sat values obtained from field measurements were on average higher than those obtained from laboratory K_sat values. Particularly for the clay texture class, much lower K_sat values were observed for laboratory (mean K_sat≈8 cm d⁻¹) compared to field (mean K_sat≈110 cm d⁻¹) measurements (Table 5). Figure 3b further illustrates the higher range of K_sat values obtained for finer texture soils (clay and loam) compared to coarser soils (sand).

Figure 4 Partial correlation between K_sat and (a) soil organic carbon OC (%), (b) bulk density (g cm⁻³), (c) clay (%), and (d) sand content (%) as heat maps. K_sat decreases with increasing clay content and bulk density, and increases with sand content. The color of each hexagonal cell shows the number of the measurements in each cell.

3.3 K_sat PTFs derivation

As a test application of SoilKsatDB, two PTFs were derived for K_sat (i.e., for measurements taken in temperate regions and based on laboratory measurements) using basic soil properties as covariates. General trends between K_sat and soil properties are shown in partial correlation plots in Fig. 4. The figure indicates that K_sat decreases with clay content and bulk density, and increases with sand content.

Figure 5 The correlation between observed and predicted K_sat values obtained from random forest (RF) models. The RF-based pedotransfer function (PTF) model was fitted using data for laboratory measurements of K_sat and tested on both laboratory (a) and field (b) measurements. Results showed reasonable agreement (CCC = 0.72) using RF algorithms for laboratory measurements, but low CCC (0.10) for field measurements. PTFs developed based on laboratory measurements do not provide accurate estimates of K_sat measured in the field.

Figure 6 Correlation between observed and predicted K_sat values obtained from the random forest (RF) model. The RF-based pedotransfer function (PTF) model was obtained by fitting 6637 training points measured in a temperate climate and tested on (a) data from temperate climate (1659 measurements) and (b) data from tropical areas (1111 measurements). PTFs showed good performance (CCC = 0.74) for the measurements taken in the temperate climate (including both laboratory and field measurements), but lower CCC values were obtained for tropical soil measurements (0.49 for RF). PTFs determined for temperate regions cannot be easily transferred to tropical regions because of the different soil forming processes.

Figure A1 shows the list of relative importance of the covariates to build PTFs for the measurements from temperate regions and laboratory-based measurements. Clay content was found to be the most important variable followed by sand and bulk density for the temperate climate PTF. On the other hand, sand content was the most important variable followed by clay and bulk density for the laboratory-based K_sat PTF. CCC, bias, and RMSE were, respectively, equal to 0.74, −0.006, and 0.64, for the temperate-based PTF, and 0.72, −0.02, and 0.66 for laboratory-based PTF.

As we will discuss in more detail in the next section, PTF models derived for temperate and laboratory-based K_sat values underestimated K_sat for tropical- and field-based K_sat values, respectively (see Figs. 6b and 5b). CCC, bias, and RMSE values were, respectively, equal to 0.49, −0.2, and 0.94 for tropical K_sat values, and to 0.10, −0.22, and 1.2 for field measured K_sat values.

4 Discussion

4.1 Laboratory vs. field estimated K_sat: effect of soil structure

The K_sat values were, on average, higher for the field measurements compared to laboratory measurements for most soil texture classes (Table 5 and Figs. 3b and 5). The difference in laboratory and field-based K_sat values and larger range of K_sat values for fine textured soil is probably related to the effect of biologically induced soil structure that might be neglected in laboratory measurements. The omission of soil structures in many laboratory samples limits the possibility to properly reproduce field observations that are likely to be more affected by the presence of biopores (Fatichi et al., 2020) and other soil structural characteristics, such a cracks. In other words, variability in the K_sat values depends on the consideration (and existence) of soil structure by the measurement methods. Soil structural pores change the pore size distribution and subsequently affect K_sat values (Tuller and Or, 2002). Such an effect is more likely to be neglected in laboratory measurements rather than in field studies due to the small size of most laboratory samples. Presence or absence of large structural pores depends on the scale of measurements (that is usually larger in the field). Mohanty et al. (1994), for example, compared three field methods and one laboratory method and found that the sample size affects the measurement of K_sat due to the presence and absence of open-ended pores. Similarly, Ghanbarian et al. (2017) showed that the sample dimensions (e.g., internal diameter and height) also impact K_sat. The authors further developed a sample dimension-dependent PTF, which performed better than other PTFs available in the literature. Likewise, Braud et al. (2017) used three field methods for K_sat measurements and found significant variation between these measurement methods. Davis et al. (1996) also highlighted the necessity to choose the most appropriate scale of measurement for a particular soil sample when undertaking conductivity measurements. They tested small cores (73 mm wide and 63 mm high) and large cores (223 mm wide and 300 mm high) using the constant head method in the laboratory and found a difference of 1 to 3 orders of magnitude.

4.2 Temperate vs. tropical soils: effect of soil formation processes

PTFs obtained for temperate soils performed poorly for tropical soils (Fig. 6), with K_sat being underestimated by the temperate-based PTFs. This result is in agreement with Tomasella et al. (2000) who derived PTFs using data from tropical Brazilian soils, which did not properly capture observations in temperate soils. We argue that the significant differences for tropical and temperate soils are due to the differences in the soil-forming processes that also define the clay type and mineralogy. In fact, Oxisols (highly weathered clay soils as a result of high rainfall and temperature in tropical regions) are characterized by inactive (non-swelling) clay minerals. In contrast to tropical soils, active (smectite) and moderately active clay minerals (illite) are the dominant clay minerals in temperate regions. These swelling clay minerals retain water within internal structures with very low hydraulic conductivity. Therefore, such a difference in clay mineralogy is likely responsible for the underestimation of K_sat in tropical soils from PTFs based on measurements in temperate areas. In addition, soil structure formation processes may be different in tropical and temperate regions (perennial activities of vegetation in the tropics), which would also lead to differences between measured K_sat values for the two climatic regions.

4.3 Limitations of SoilKsatDB

We put an effort to combine laboratory and field data from across the globe. However, we acknowledge that there are still gaps in some regions, such as Russia and higher northern latitudes in general, which may result in uncertainties in K_sat estimates in such regions. The SoilKsatDB could also be of limited use for fine-resolution applications because many data points were characterized by limited spatial accuracy and missing soil depth information. Specifically, the spatial accuracy of many points is between tens of meters and several kilometers (see the methodology sections regarding the extraction of the spatial locations using Google Earth). Many of the records in the SoilKsatDB come from legacy scientific reports and the original authors could not be traced and contacted, hence we advise to use these data with caution. In addition, in the SWIG database, the soil depth and measurement method information were not provided, and often one location was used to represent an entire watershed. We tried to revisit each publication and extract the most accurate coordinates of assumed sampling locations. In addition, we assumed that most of the samples were obtained from field measurements as authors used different infiltrometers to compute K_sat, so there might be a few points in our SoilKsatDB that belong to laboratory measurements and that we have incorrectly assigned to field measurements. Moreover, the field measurements in the database are a mix of many K_sat measurement methods.

For each measurement, a location accuracy (0–100 m = highly accurate, >10 000 m = least accurate) was assigned based on the sampling location accuracy. The location accuracy can be used as a weight or probability argument in machine learning for K_sat mapping. We are aware that this was a rather subjective decision; a more objective way to assign weights would be to use the actual spatial positioning errors. Because these were not available for most of the datasets, we have opted for the definition of a location accuracy estimated from the available documentation.

4.4 Further developments

The advancement in remote sensing technology opens the doors to link the hydraulic properties with global environmental data. Satellite-based maps of environmental characteristics such as local information on vegetation, climate, and topography for specific areas, which are often ignored by basic PTFs, can be incorporated. For example, Sharma et al. (2006) developed PTFs using environmental variables such as topography and vegetation and concluded that these attributes, at fine spatial scales, were useful to capture the observed variations within the soil mapping units. Likewise, Szabó et al. (2019) used a random forest machine learning algorithm for mapping soil hydraulic properties and incorporated local environmental information such as vegetation, climate, and topography.

5 Data availability

All collected data and related soil characteristics are provided online for reference and are available at https://doi.org/10.5281/zenodo.3752721 (Gupta et al., 2020). Please note that our zenodo collection of files contains more than is described in the paper.

6 Summary and conclusions

We compiled a comprehensive global dataset of K_sat measurements (N=13 258) by importing, quality controlling, and standardizing tabular data from existing soil profile databases and legacy reports, as well as the scientific literature. The SoilKsatDB covers a broad range of soil types and climatic regions and hence is useful in global models. A larger variation in K_sat values was observed for fine-textured soils compared to coarse-textured soils, indicating the effect of soil structure on K_sat. Moreover, K_sat values obtained from field measurements were generally higher than those from laboratory measurements, likely due to the impact of soil structural pores in field measurements.

The new database was used to develop PTFs using RF algorithms for K_sat values obtained for temperate climates and for laboratory measurements. PTFs developed for a certain climatic region (temperate) or measurement method (laboratory) could not be satisfactorily applied to estimate K_sat for other regions (tropical) or measurement method (field) due to the role of different soil forming processes (inactive clay minerals in tropical soils and impact of biopores in field measurements).

There are still some gaps in the geographical representation of the data, especially in Russia and the higher northern latitudes, that could induce uncertainty in global modeling. Therefore, the dataset can be further improved by covering the missing areas thus allowing better accuracy in modeling applications.

The SoilKsatDB was developed in R software and is available via https://doi.org/10.5281/zenodo.3752721. We have made the code and data publicly available to enable further developments and improvements.

Appendix A

Table A1 Listed p values under 95 percent confidence interval for each class to show the significant difference in K_sat between texture classes (see Fig. 3b). The significant difference was computed using ANOVA (analysis of variance) with post hoc Tukey's HSD (honestly significant difference). The values in bold show the significant difference in K_sat between two soil texture classes.

Table A2 Listed p values under 95 % confidence interval for each class to show the significant difference in K_sat between texture classes for Fig. 3d. The significant difference was computed using ANOVA (analysis of variance) with post hoc Tukey's HSD (honestly significant difference) test. The values in bold show the significant difference in K_sat between two soil texture classes.

Figure A1 Importance of the variables for developing the PTFs for K_sat using the random forest algorithm. The x axis displays the average increase in node purity (the larger the value, the more important a covariate). (a) Clay content was the most important variable followed by sand and bulk density for the random forest model built on data from temperate regions. (b) Sand content was the most important variable followed by clay and bulk density for the random forest model based on laboratory measurements.