The GTN-P Database (GTN-P, 2015) is accessible online at http://gtnpdatabase.org or through the GTN-P website at http://www.gtnp.org. The general framework of the GTN-P Data Management System (DMS) is based on open source technologies following an object-oriented data model (Fig. 2) implemented with CakePHP and the database PostGIS, the spatial version of PostgreSQL (Obe and Hsu, 2011). The database distinguishes between permafrost temperatures and annual thaw depths (i.e., active layer depths). To ensure interoperability and enable inter-database searching, metadata field names are based on a controlled vocabulary registry. The documentation of the DMS is available and regularly updated on gtnp.org (ISSN 2410-2385) as the database framework and content evolves.

The online interface of the GTN-P Database was developed to maximize usability both for the data provider and user. The resulting roles (data administrator, data provider and data user) are built into the database, providing different rights to read, edit or modify data. Data users can access the database without an account and password and have access to (i) permafrost temperatures, (ii) annual thaw depths and (iii) help sections. While administrators have full access, data providers cannot modify or delete data of third parties. Data not marked as “published” by the data providers are not accessible to third parties or the public. The help section provides tutorials and template files for upload and download of borehole temperature and active layer grid data as well as GTN-P maps and fact sheets.

GTN-P follows an open-access policy in line with the IPY data policy and the GEO (Group on Earth Observations) data-sharing principles. The GTN-P Steering Committee decided on a general embargo period of 1 year. This means that data from 2015 will be available at the earliest in 2016 in order to allow investigators the first opportunity to publish their data. For special cases, e.g., doctoral dissertations, this embargo may be extended on demand. The data will be made freely available to the public and the scientific community with the belief that their wide dissemination will lead to greater understanding and new scientific insights because global scientific problems require international cooperation. Data download is unrestricted and requires only a free registration needed for web security reasons. Before being able to download data, users must accept the terms and conditions of the data use policy. Therein, the user is asked to contact the site PIs prior to publication to prevent potential misuse or misinterpretation of the data. In addition, an email is automatically sent to the contact person of each data set downloaded to inform them of the interest in the data.

A thorough data mining effort was conducted prior to the creation of the GTN-P Database. The recovered data sets were characterized by an extreme diversity. These included global data sets on active layer thickness from the CALM data collection (Shiklomanov et al., 2008), as well as data sets aggregated thematically, geographically or institutionally. Data providers include the Advanced Cooperative Arctic Data and Information Service (www.aoncadis.org) at the National Snow and Ice Data Center (http://nsidc.org), the Permafrost Laboratory (University of Alaska Fairbanks), NORPERM (Juliussen et al., 2010) and PERMOS (PERMOS, 2013), among others. Part of the data was provided by individual permafrost research groups and relayed into the database by the GTN-P national correspondents (NCs). In addition to GTN-P standard data sets on temperature and active layer thickness, several ancillary existing data sets were opportunistically added to the database. These include in particular remotely sensed land surface temperature and surface soil moisture values that were transferred from ESA DUE Permafrost (Bartsch and Seifert, 2012; DUE Permafrost Project Consortium, 2012).

The data upload procedure was conceived to eliminate the need for any prior knowledge of databases by the user. NCs from all participating countries were nominated by the national committees and by the scientific international permafrost community to provide data on an annual basis, collecting information from the investigators and data managers from that country. NCs are listed on the GTN-P website and can be contacted by permafrost researchers interested in contributing monitoring data to the GTN-P Database. NCs are also encouraged by the GTN-P Steering Committee to pro-actively engage national investigators in the process to ensure a continuous upload of data into the system. The requirements of the data upload are compliant with existing international standards for geospatial metadata ISO 19115/2 and TC/221 (www.iso.org). The database specifically builds on the GTN-P metadata form that was developed as a standard by the GTN-P leadership in 1999 (Burgess et al., 2000). Site metadata must be entered and selected from parameters and properties, which are selectable in drop-down lists in the upload interface. GTN-P metadata used in this paper are available at 10.1594/PANGAEA.842821 (GTN-P, 2015). Tutorials and templates of data files provide the necessary information to bring the data into the right CSV (comma-separated values) format, prior to data upload. The maximum file size for upload is 1.5 MB.

The GTN-P Database features both (i) basic search and (ii) custom search functions. The goal of these functions is to narrow down the number of data records based on a set of criteria. While the basic search is a simple filter by manual character input, the advanced custom search allows the use of multiple search criteria to retrieve a defined list of data records from the repository. The data and metadata associated with the search results can be downloaded by the data user as compressed file packages containing standardized metadata forms in text and XML (Extensible Markup Language) and the corresponding data in CSV format. However, the CSV format and the inconsistency of the time series, in regard to completeness, frequency and geometry, do not facilitate their direct use within climate models, as they do not comply with the CF (Climate and Forecast) 1.6 convention. Therefore, the GTN-P as well as other global terrestrial networks should link processes, structures and technologies with existing marine and atmospheric networks, aiming at close cooperation for their assimilation into climate models. Data heterogeneity in terms of spatial variability, frequency, measurement profile and methods complicates data gridding. To address this issue, we developed tools for data analysis, processing and quality assurance, through the data model definition and a set of internal database triggers, functions and a SQL-to-NetCDF Python converter. Measurement frequencies and methods, sampling profiles, values and null values count, time series first and last date, time gaps, total and annual minimum, maximum, average, standard deviation and variance are systematically retrieved during the data upload. TSP data sets are linearly interpolated at consistent 0, 1, 2, 3, 5, and 10 m borehole depths. All eligible data sets are aggregated into a NetCDF (Network Common Data Form) file. Conventions for CF 1.6 metadata are designed to promote the processing and sharing of files created with the NetCDF Application Programmer Interface and provides a definitive description of the spatial and temporal properties of the data. The resulting NetCDF files represent (i) a TSP data set in a multidimensional array representation of annual time series profiles of ground temperature, orthogonal along vertical and orthogonal along time, and (ii) a CALM data set in an orthogonal multidimensional array representation of annual times series of active layer thickness.

At the time of submission of this paper the GTN-P Database contained metadata from 1074 TSP boreholes and 243 ALT monitoring sites, out of which 239 are taken from the CALM database. Thirty-one boreholes are located in the mountain permafrost regions and 72 in Antarctica. Currently, 277 borehole sites have temperature data and 78 active layer monitoring sites have annual thaw depth data. Due to the fact that one site can have more than one measurement unit or period, the total number of data sets differs from the number of sites: 1062 ground temperature data sets (including surface and air temperature); 78 active layer thickness data sets; and 160 extra data sets with surface soil moisture from satellite measurements, boreholes, and active layers sites.

In addition to the data quality control of the individual permafrost scientist, the GTN-P Data Management System offers quality control. Data being entered into the database undergo several steps of quality checking before receiving approval for data output. To harmonize the different data formats and produce one standard format within the GTN-P Database, every data set retrieved from external sources underwent a review and a standardization procedure to bring the file into the correct format for upload. This includes, in particular, conversion of file structures, date formats, reference points and null values. Metadata input must be compliant with the database rules and include a number of mandatory fields employing terminology code lists and controlled vocabulary associated with the GTN-P Database (Fig. 2) as documented on the GTN-P website. Successful upload assures correctness and consistency of the data set. Screening for obvious errors follows with the help of automated data visualization during and after the upload procedure. Interactive and adjustable data plots on the database website serve also as on-the-fly data visualization for scientific purposes. According to the GTN-P Strategy and Implementation Plan 2012–2016 (GTN-P, 2012) metadata and data considered for input into the GTN-P Database will be coordinated and reviewed by NCs on a regular basis, at least once per year. Quality checked data sets are published only after approval by the NCs, who are responsible for the quality assessment of permafrost data from their countries.

In geoscience, errors start to emerge as early as at the measurement stage. The most common technique of continuously recording borehole ground temperatures at specific depths is the use of permanently installed multi-thermistor cables, providing an accuracy and precision between ca. 0.02 and 0.1 ∘C (Brown et al., 2000; Romanovsky et al., 2010b). The logger resolution and measurement frequency, however, varies with the type and the depth of the individual borehole. Due to active layer dynamics, the relative vertical position of measurement probes can change and hence introduce an error into the depth indications of old boreholes in sensitive areas. Additionally, the number of vertical positions of sensors varies not only between and within boreholes (and research groups) but also through time. Commonly, sensors are placed every 0.2–0.4 m until 2 m depth, every 0.5 m until ca. 4–5 m depth, and every 1–3 m until 15 m depth, and in the deeper parts of a borehole sensor they are reduced to 5–10 m steps (Brown et al., 2000). The most frequent sensor depth found in the GTN-P TSP data sets is 5 m, followed by 1, 3 and 10 m in decreasing order. A linear regression on 180 data sets from 158 boreholes indicates that the overall number of temperature sensors increase with increasing borehole depth. Within the boreholes, the deviation of the numbers of temperature recordings per time unit ranges from 0 to 57 measurements. Due to the applied methods and demanding field work conditions in permafrost areas, the number of measurements per time vary not only from borehole to borehole but also between time series from the same borehole.

To provide sufficient information about the data quality by using the possibilities of the database, this inconsistency must be assessed both qualitatively and quantitatively. Therefore, the GTN-P Secretariat established an IPA action group to develop strategies for profound numerical assessment and control of the GTN-P data quality, which are being discussed at GTN-P NC workshops and in scientific articles (Biskaborn et al., 2015). The GTN-P database at present stage does not apply fully developed quality criteria, meaning that some of the sites will be revised according to the outcome of the IPA action group.

ALT data generally have fewer numbers of potential biases due to the majority of sites performing measurements of summer thaw depths using mechanical probing either in grids or transects, resulting in multiple measurements compared to point locations associated with sites using thaw tubes or temperature boreholes.

We assessed the overall metadata completeness for TSP and ALT data sets by calculating the percentage of available fields that are filled in. Figure 3 indicates the percentages of both data types according to the metadata completeness and the establishment (age) of the monitoring site. ALT metadata are generally more complete, with values between 50 and 80 % (average 63 %). TSP shows a clustered distribution of metadata completeness with most data sets between 31 and 40 % and a second maximum between 61 and 70 % (average 50 %). Metadata fields (Fig. 2) with the most missing information are accessibility, distance from disturbance, bibliographic references, terrain morphology, hydrology, slope and aspect, borehole diameter, and permafrost thickness. While these “extra” information are not essential for the direct permafrost monitoring, they are relevant to gain a holistic future view on the thermal state of permafrost by feeding high-quality data to global models.

The complex nature of ALT grid metadata, however, created inconsistencies in the structure of the primary data files. Even though the files were standardized before implementation, the low resolution of a number of CALM grid references and TSP borehole coordinates led to imprecise geopositioning: 275 longitudes (20.4 %) and 287 latitudes (21.0 %) have less than four decimal places. A total of 374 data sets (27.8 %) had coordinates with decimal degree precision below four decimal places of either the latitude or the longitude or both. While these data sets are being successively revised by the NCs, coordinates will be allocated more precisely.

We divided the GTN-P borehole depth classes into 1 m bins after Burgess et al. (2000). As Fig. 7 shows, the greatest number (42.3 %) of all TSP boreholes belong to the surface class (“SU”, < 10 m). In general, there are more shallow (30.6 % “SH”, 10–25 m) and intermediate boreholes (17.8 % “IB”, 25–125 m) than deep boreholes (9.3 % “DB”, > 125 m). The peaks in the borehole depth distribution correspond to commonly chosen depths (3, 5, 10, 15, 20, 25 and 30 m). These were often defined prior to drilling to capture specific permafrost features such as the depth of zero annual amplitude (DZAA). Deep boreholes are generally older than shallow boreholes. The average drilling dates (year) for the GTN-P depth classes are as follows: SU, 2003; SH, 1997; IB, 1993; DB, 1984. The overall average drilling date of boreholes is 1997. However, only 82 % of TSP data sets contain metadata information about borehole ages. The lack of age metadata affects all depth classes. The average borehole depth of all sites is 53 m, and that of data sets without age information is 29 m. The oldest borehole currently present in the database is located in Russia (Vorkuta K-887) and was drilled to 85 m depth in 1957.

Table 1 summarizes the distribution of boreholes and active layer monitoring sites per country. The total numbers per country and permafrost zone were calculated by plotting the sites as points and the areas as polygons in ArcGIS. During the analyses some polygons and site coordinates suffered from inaccuracy – for example, terrestrial boreholes with imprecise coordinates were shown as “offshore” sites. In these cases, land–ocean polygon boundaries were slightly shifted and the land polygons extended to capture the relevant points. For calculating the borehole per area ratios, however, we used the original polygon dimensions. The NCs will refine imprecise coordinates during revision of the metadata of their countries. The sites are still included in the analysis due to negligible bias on a hemispheric or Arctic scale.

In order to measure the degree of inhomogeneous sampling and to identify the main geographical gaps with focus on the Arctic, we performed a numerical quantification of the distribution of boreholes and active layer grids in the Northern Hemisphere with the help of a Voronoi tessellation analysis (VTA) as suggested by Molkenthin et al. (2014). This analysis demonstrates the potential of the GTN-P Database for assessing the monitoring of permafrost on a hemispheric scale. However, the metadata quality must be successively improved with the help of the NCs and more data must be provided from the international permafrost community to gain a complete and accurate evaluation.

To reduce the potential bias that results from multiple boreholes or active layer monitoring grids around the same coordinate or which are very close to each other, buffers of 1 km radius for each coordinate were created in ArcGIS. Sites with site-to-site distance of ≤ 2 km were merged and the gravitational centers of the resulting buffer areas were converted to points for further calculations. With the help of this method we reduced 1074 TSP coordinates to 614 buffered TSP sites and 243 ALT coordinates to 187 buffered ALT sites. Glaciated areas (shapefile from Natural Earth data, 50 m resolution) were removed from the analysis. Voronoi cells were calculated using the Thiessen polygon tool and subsequently clipped to the extension of the IPA map of permafrost zones.

The VTA creates a mosaic by drawing area (cell) boundaries exactly in the middle between neighboring nodes: TSP sites (Fig. 4) and ALT sites (Fig. 5). Every point within a cell is closer to its node than to any other node. In a VTA, uniform distribution of sites would result in maximum peak in the cell size distribution at the same value as Atotal/Ncells (Molkenthin et al., 2014), which is basically the same as the mean Voronoi cell size. Hence, to quantify the overall deviation from equidistant sampling of the terrestrial Northern Hemisphere permafrost and glacier-free area, we used the standard deviation (SD) of the Voronoi cell size distribution from TSP (SD: 9.08 × 104 km2) and ALT (SD: 8.68 × 104 km2). For visualization, we calculated the number of Voronoi cells in a cubic size sequence x2 (1 to 2, 2 to 4, 4 to 8,... , 1.05 × 106 to 2.10 × 106 km2) and plotted the results on a logarithmic scale (Fig. 6). Voronoi cell size ranges were attributed to the same color types as in Figs. 4 and 5. According to the VTA, the TSP cell size distribution peaks two times at smaller values than the Atotal/Ncells= 3.79 × 104 km2, indicating a significantly clustered sample distribution. TSP bimodal size distribution is attributed to (i) linear spatial sample configuration along transportation corridors in areas with developed economic and infrastructure as well as several high-density borehole transects and (ii) to the good coverage and high number of boreholes in Alaska, both indicated in green. The ALT cell size peaks at about the same values as Atotal/Ncells= 1.25 × 105 km2. The plateau between 2 × 104 and ca. 6 × 105 km2, however, indicates a clustered sample distribution, although the panarctic ALT sampling is clustered to a lesser degree than the borehole configuration. High skewness of both TSP (4.99) and ALT (6.52) cell size distributions indicates that the peaks are inclined towards higher cell size values demonstrating inhomogeneous sample distribution.

The boundaries of the bigger Voronoi cells (orange and red) and especially their intersections (Figs. 4, 5 and 8) indicate locations with the highest potential for improving the representativeness of permafrost monitoring from hemispherical perspective. Furthermore, these methods can facilitate the planning and establishment of new permafrost monitoring sites in the context of climate change by establishing boreholes and ALT sites where lines intersect with high soil organic carbon content and the transitional zone from continuous to discontinuous permafrost. However, this statement is based on a purely statistical view of the Northern Hemisphere and is not taking into account disturbances resulting from, for example, water bodies, forest fires, infrastructure, areas of deforestation, urbanization, farming, mining and wetland drainage.

Together with the TSP and ALT Voronoi cell boundaries and simplified permafrost zones, we illustrated the panarctic distribution of soil organic carbon content within the top two meters by using data from the Northern Circumpolar Soil Carbon Database (Hugelius et al., 2013) in Table 2. The distribution of ALT and TSP point coordinates was calculated within the different carbon content groups and shows that, at the circumpolar scale, 25.2 % of all boreholes and almost 29 % of all ALT sites are located in permafrost areas that contain more than 25 % organic carbon. However, only 1.7 % of the boreholes and zero ALT sites cover areas with more than 50 % organic carbon.

We conducted a similar analysis using vegetation zones. For this, we used the vegetation zone information provided in the original GTN-P metadata. Locations with missing vegetation information were attributed to vegetation zones by using photographs of the site (if available) and/or other sources such as atlases of the local flora. This information is provided in Table 2. Because of the wide variety of sources used to define vegetation zones, we prefer not to base recommendations for future locations of monitoring sites based on this information. However, the treeline (Walker et al., 2005), through its function as a major ecotone between forest and tundra, offers high potential for sensitive recording of climate change signals (e.g., Biskaborn et al., 2012) and is therefore shown in Fig. 8.

Topography and in particular slope orientation influence the amount of solar radiation received by the ground surface and the accumulation of snow. Due to orbital parameters, in mountainous regions of lower latitudes, permafrost occurs preferably on north-facing slopes in the Northern Hemisphere. Similarly, in continuous permafrost regions, the active layer is usually thinner on north-facing slopes (French, 2007). To inspect the monitoring bias that might be caused by preferential slope orientation, we analyzed the slope and aspect for boreholes and active layer sites.

Only few of the original GTN-P metadata collections contained slopes and aspects of the ground surface at the permafrost borehole or the active layer grid sites. This information also existed in various formats. We used the ESA DUE Permafrost Circumpolar digital elevation model (Santoro and Strozzi, 2012) in ArcGIS to calculate slope and aspect from the Northern Hemisphere topography. This remote-sensing-derived model, however, has a resolution of 100 m, and therefore the calculated values (in degree units) for each site north of 60∘ N should be evaluated carefully. Figure 9 shows the slopes and aspects and their statistics for the original metadata and the calculated values in spherical projections plotted with STEREONET 9.2 (Cardozo and Allmendinger, 2013). The graph includes both surface areas at TSP and ALT sites as (i) planes in equal-angle projections and (ii) the frequencies of slope aspects as rose diagrams with a bin size of 30∘. A comparison between the original metadata and the DEM-derived values shows major differences in the ALT sites, amplified (i) by the very low number of slope metadata entries (n=8) and (ii) due to the fact that most active layer monitoring sites are located on a more or less flat terrain. It must be considered, however, that the majority of ALT sites are established in zonal geophysical conditions typically found on flat watersheds and only a few historically adapted sites are located on slopes or in complex terrain.

The closer the slope values are to zero, the higher the potential uncertainty in the aspect values. Aspects in the original TSP and ALT metadata had various formats including verbal descriptions and abbreviations of main (rough) geographical directions. Accordingly, these rose diagrams and planes are concentrated in categorized directions such as N, NW and WNW. A higher overall number of TSP borehole slopes and aspects (n=48) from the metadata than for ALT sites enabled a more reliable comparison between original and calculated values. For all TSP sites north of 60∘ N, 25 % of the original metadata and 20 % of the DEM-derived data rank in the bin between 271 and 300∘. Both mean vectors point towards a WSW direction, as indicated in Fig. 9 by the arrows. The fact that slopes at the borehole and ALT sites are dipping towards a preferential direction indicates that there is a different amount of incoming solar energy received by the monitored ground than compared to the average. Therefore, preferential slope orientation causes a bias in the overall representativeness of temperature monitoring and should be taken into account when using the data for global models.

Climate models project temperature increases in the Arctic towards the end of the 21st century that are larger than anywhere else on Earth (ACIA, 2004; IPCC, 2013). CMIP5 models show that, for each degree of global temperature increase, about 1.6 × 106 km2 or ca. one-quarter of the present near-surface permafrost area is expected to start to disappear (Koven et al., 2013) and boreal landscapes will most likely lose all present discontinuous permafrost zones by the end of the 21st century (Slater and Lawrence, 2013). To assess the distribution quality of present permafrost temperature monitoring, we calculated the number of TSP and ALT sites per zone of projected temperature change for 15 different climate models. We created maps showing the spatial distribution of the mean annual near-surface temperature change (T 2070–2099 minus T 1970–2000) for representative concentration pathways (rcp's) 4.5 and 8.5 using models: ACCESS1-0, bcc-csm1-1, CanESM2, CCSM4, CNRM-CM5, CSIRO-MK3-6-0, GISS-E2-H, GISS-E2-H, GISS-E2-R, HadGEM2-ES, inmcm4, IPSL-CM5A-LR, MPI-ESM-LR, MRI-CGCM3 and NorESM1-M. Thereafter we calculated the number of GTN-P sites per 1∘C sector. Figure 10 shows that in rcp 4.5, an intermediate greenhouse gas emission scenario, most boreholes and ALT sites are located in relatively narrow zones of less extreme projected temperature change (ca. 3–6 ∘C for TSP and ca. 2–5 ∘C for ALT). The high-emission scenario rcp 8.5 projects a more extreme temperature increase for larger areas and more GTN-P monitoring sites are located in zones of up to a 10 ∘C potential temperature rise. A comparison of the applied models shows that, depending on the model uncertainties and variety of possible climate futures, the spatial distribution of projected temperature change varies from model to model. This is why increasing the number of soil temperature and active layer monitoring sites by filling main geographical gaps is critically important to constrain projections of climate change's impact on permafrost.