Although they have become the focus of intense research activity in recent times, with more papers published in the 5 years prior to 2018 than all years before that combined, machine learning (ML) techniques applied to geoscience problems date back to the middle of the last century (see Van der Baan and Jutten, 2000, and Dramsch, 2020, for reviews). ML applications relate to a whole range of geological and geophysical problems, but many of these studies face common challenges due to the nature of geoscientific data sets. Karpatne et al. (2017) summarise the principal challenges as follows: i.

Objects with amorphous boundaries. The form, structure and patterns of geoscience objects are much more complex than those found in the discrete spaces that ML algorithms typically deal with, consisting of changes in both the topology and the dimensionality of geoscience objects with time.

We address issues related to multi-resolution data by providing a “controlled” data set, at the same resolution; it offers possibilities of addressing multi-resolution issues, by subsampling or upscaling.

We address noise, incompleteness and uncertainty in data by providing synthetic data; we have noise- and uncertainty-free data, or at least under control, and complete spatial coverage over the simulation domain. The models we provide can easily have a structured or unstructured noise added to them, and they can be subsampled to reproduce incomplete data sets.

We address small sample size by generating 1 million models, which is certainly not enough to thoroughly explore the high dimensional model space; however, it illustrates the feasibility of producing large suites of models in the near future. Modern ML training sets for popular subjects such as the human face may contain tens of millions of examples (Kollias and Zafeiriou, 2019). A search of the Kaggle database of training data sets (https://kaggle.com, last access: 27 January 2022, which contains over 63 000 distinct data sets at the time of writing) had only 151 data sets with geoscience in the keywords, and only seismic catalogues featured as geophysical data. Similarly, only 59 data sets contained 3D data, and none were related to the geosciences.

Finally, we address the spatial and temporal paucity of ground truth by publishing over 1 million models for which the full 3D lithological and petrophysical distribution is provided in a labelled form for comparison with resulting gravity and magnetic fields. This challenge is also faced by geophysical inversion methods. Three-dimensional geological models built using sufficient data to reduce uncertainty arguably exist, but leaving aside a strict definition of uncertainty, well-constrained 3D geological models are primarily restricted to restricted areas of significant economic interest, specifically sedimentary basins and mineral deposits, which only represent a subset of possible geological scenarios. A number of studies have built simple or complex synthetic models as a way to overcome these problems by providing fully defined test cases for testing processing, imaging and inversion algorithms (Versteeg, 1994; Lu et al., 2011; Salem et al., 2014; Shragge et al., 2019a, b). Whilst these provide valuable insights, the efforts required to build these test cases preclude the construction of large numbers of significantly different models. It is easy enough to vary petrophysical properties with fixed volumes; however varying the geometry and, in particular, the topology is time-consuming.

Implicit geological modelling is based on the calculation of scalar fields that can be iso-surfaced to retrieve stratigraphy and structure, as opposed to earlier methods that were CAD-like or based on the interpolation of data points. Recent advances in implicit modelling allow extensive geology model suites to be generated by perturbing the data inputs to the model (Caumon, 2010; Cherpeau et al., 2010; Jessell et al., 2010; Wellmann et al., 2010; Wellmann and Regenauer-Lieb, 2012; Lindsay et al., 2012, 2013a, b, 2014; Wellmann et al., 2014, 2017; Pakyuz-Charrier et al., 2018a, b, 2019) as part of studies that characterised 3D model uncertainty; however since they use a single model as the starting point for the stochastic simulations, these works do not provide a broad exploration of the range of geological geometries and relationships found in nature. Work on the automating of modelling workflows may allow us to explore the model uncertainty space more efficiently (Jessell et al., 2021).

In this study, we have created a massive open-access resource consisting of 1 million three-dimensional geological models using the Noddy modelling package (Jessell, 1981; Jessell and Valenta, 1996). These are provided as the input file that define the kinematics, together with the resulting voxel model and gravity and magnetic forward-modelled response. The models are classified by the sequence of their deformation histories, thus addressing a temporal paucity of ground truth. This resource is provided to anyone who would like to train a ML algorithm to understand 3D geology and the resulting potential-field response or to anyone wishing to test the robustness of their geophysical inversion techniques. Guo et al. (2022) used the same modelling engine to produce more than 3 million models of a more restricted range of parameters to train a ML convolutional neural network system to estimate 3D geometries from magnetic images. In this study we aim to provide a much broader range of possible geological scenarios as the starting point for a more general exploration of the geological model space.

The Noddy software has been used in the past for a range of studies due to its ease in producing “reasonable-looking” geological models with a low design or computational cost. A precursor to this study used 100 or so manually specified models as a way of training geologists in the interpretation of regional geophysical data sets by providing a range of 3D geological models and their geophysical responses (Jessell, 2002). Similarly, Clark et al. (2004) developed a suite of ore deposit models and their potential-field responses. The automation of model generation using Noddy was first explored using a genetic algorithm approach to modifying parameters as a way of inverting for potential-field geophysical data, specifically gravity and magnetics (Farrell et al., 1996). Wellmann et al. (2016) developed a modern Python interface to Noddy to allow stochastic variations of the input parameters to be analysed in a Bayesian framework. Finally Thiele et al. (2016a, b) used this ability to investigate the sensitivity of variations in spatial and temporal relationships as a function of variations in input parameters.

In this study we draw upon the ease of generating stochastic model suites to build a publicly accessible database of 1 million 3D geological models and their gravity and magnetic responses.

The Noddy package (Jessell, 1981; Jessell and Valenta, 1996) provides a simple framework for building generic 3D geological models and calculating the resulting gravity and magnetic responses for a given set of petrophysical properties. The 3D model is defined by superimposing user-defined kinematic events that represent idealised geological events, namely base stratigraphy (STRAT), folds (FOLD), faults (FAULT), unconformities (UNC), dykes (DYKE), plugs (PLUG), shear zones (SHEAR-ZONE) and tilts (TILT), which can be superimposed in any order, except for STRAT, which can only occur once and has to be the first event. The 3D geological models are calculated by taking the current x,y,z position of a point and unravelling the kinematics (using idealised displacement equations) until we get back to the time when the infinitesimal volume of rock was formed, whether defined by the initial stratigraphy, the time of formation of a stratigraphy above an unconformity or an intrusive event. In this study, we use only the resulting voxel representation of the 3D geological models; however it is possible to produce iso-surface representations of the pre-deformation location of points in an implicit scheme. We have used this tool as it is rapid, taking under 15 s to generate 200×200×200 voxel models with both geological and geophysical representations combined using an Intel^® Xeon^® Gold 6254 CPU at 3.10 GHz, and produces “geologically plausible” models that may occur in nature. Given that the final 3D model depends on the user's choice of geological history, Noddy can be thought of as a kinematic, semantic, implicit modelling scheme.

As opposed to Wellmann et al. (2016), Thiele et al. (2016) and Guo et al. (2022), who used a Python wrapper to generate stochastic model suites, in this study we have modified the C code itself to simplify use by third parties, although the philosophy of model generation is an extension of, as well as very similar to, these earlier studies. The most significant difference is that we have added petrophysical variations by randomly selecting from a set of stratigraphic groups; see the next section.

Figure 1 shows one example model set for a STRAT–TILT–DYKE–UNC–FOLD history, consisting of a 3D visualisation looking from the NE of the voxel model, with some units rendered transparent for clarity; the top surface of the model; an E–W section at the northern face of the model looking from the south; a N–S section on the western face of the model looking from the east; and the resulting gravity and magnetic fields.

In this section we describe the choices and range of values for the parameters that we have allowed to vary for our 1-million-model suite. We recognise there are other unused modes of deformation that Noddy allows that have been ignored. The selection of these parameters is based on assessing the range of parameter values that will produce suites of models that we believe will help in and not hinder addressing the challenges cited in the Introduction. For example, we limited the size of the plugs so that a single plug could not replace the geology of the entire volume of interest. In the discussion, we refer to additional event parameters that could be activated in future studies. We limited the study to five deformation events, starting with an initial horizontal stratigraphy which is always followed by tilting of the geology. The following three events are drawn randomly and independently from the event list comprised of folds, faults, unconformities, dykes, plugs, shear zones and tilts. The likelihoods of folds, faults and shear zones are double those of the other events as we found, based on a qualitative assessment, that they had a bigger impact of changing the overall 3D geology, and hence we wished to sample more of these events. This means we can have 73=343 distinct deformation histories, although the specific parameters for each event can also vary, so the actual dimensionality of the system is much higher. For clarification, the 1 million models are not the result of a combinatorial approach but of 1 million independent draws using a Monte Carlo (MC) sampling of the model space. Whilst a combinatorial approach may in theory explore the parameter space more uniformly, the sequence of five deformation events is so non-linear that it was reasoned that a pure MC approach would serve our purposes.

The initial stratigraphy, as well as new, above-unconformity stratigraphies, is defined to randomly have between two and five units to keep the systems relatively simple, but this could of course be increased if desired. The lithology of each unit in a stratigraphy is chosen to be coherent with the specific event and other units in the same sequence so that we do not, for example, mix high-grade metamorphic lithologies and un-metamorphosed mudstones in the same stratigraphic series (Table 2), nor do we assign the petrophysical properties of a sandstone to an intrusive plug. Once a lithology is chosen, the density and magnetic susceptibility are randomly sampled from a table defining the Gaussian distribution of properties (linear for density, log-linear for magnetic susceptibility) for that rock type. In the case of densities this may result in occasional negative values; however since the gravity field is only sensitive to density contrasts, this does not invalidate the calculation. Some rock types have bimodal petrophysical properties to reflect real-world empirical observations, so we draw from a Gaussian mixture in these cases. The petrophysical data are drawn from aggregated statistics (mean and standard deviation of one or two peaks) of the approximately 13 500-sample British Columbia petrophysical database (Geoscience BC, 2008).

The parameters which can be varied for each type of event, together with the range of these parameters, are shown in Table 1. These parameters can be grouped by the shape, position, scale and orientation of the events, and for a five-stage deformation history, the random selection is required of a minimum of 23 parameters for a STRAT–TILT–TILT–TILT–TILT model and up to 69 parameters for a STRAT–TILT–UNC–UNC–UNC model where each stratigraphy has five units. Apart from the petrophysical parameters, all other parameters are randomly sampled from a uniform distribution.

Any subset of the geology can be calculated for any sub-volume of an infinite Cartesian space using Noddy, but we limit ourselves to a 4×4×4 km volume of interest in this study. Similarly, although the geology within this volume can be calculated at an arbitrary resolution, we have chosen to sample it using equant 20 m voxels as this is well below the typical resolved measurement scale for these types of data when collected in the field.

Geophysical forward models were calculated using a Fourier domain formulation using reflective padding to minimise (but not remove) boundary effects. The forward gravity and magnetic field calculations assume a flat top surface with a 100 m sensor elevation above this surface and the Earth's magnetic field with vertical inclination, zero declination and an intensity of 50 000 nT.

The 73 possible event histories produce 343 possible sequences, which averages to 2915 models per sequence. Given the imposed bias towards folds, faults and shear zones, different event sequences were more or less likely to be found in the 1-million-model suite. The high-probability event sequences (e.g. FAULT–SHEAR ZONE–FOLD) produced 8245 models, while the low-probability event sequences (e.g. UNC–TILT–PLUG) produced only 905 models. The different combinations produced plateaux in the number of models calculated, giving event sequence frequencies at around 1000, 2000, 4000 and 8000 depending on the number (0, 1, 2 and 3 respectively) of events in the sequence. Together these form a “Noddyverse” of 1 million 3D geological models and their gravity and magnetic responses. Figure 2 shows an arbitrarily selected suite of 100 models as a 10×10 grid showing the top surface and two sections of the model as in Fig. 1, together with the resulting gravity and magnetic fields, to show the variability in the results.

The same logic of using millions of Noddy models was first applied by generating a massive 3D model training set and used to invert real-world magnetic data (Guo et al., 2022). That study used a model suite consisting of only FOLD, FAULT and TILT events and only one of each to predict 3D geology using a convolutional neural network (CNN). This approach corresponds to a use case where prior geological knowledge of the local geological history has been used to limit the model search space, and formal expert elicitation could provide an important precursor step to support the generation of sensible and tractable problems. In addition to the CNN training demonstrated by Guo et al. (2022), we can envisage three broad categories of studies that could build upon the 3D model database we present here.

Studies into the uniqueness of 3D models relative to geological event histories. The principal question here is whether any form of classification of the patterns seen in the geophysical fields, perhaps including mapping of the surface, can be used to recover the event sequence or event parameters. Feature extraction techniques are well known for supporting image classification and clustering, so using the same principles, can we identify unique clusters of forward models from the Noddyverse, and do these clusters then correspond to distinct histories? Likewise, can we train a classifier with extracted features from the forward models of the gravity and magnetic responses which can then successfully identify models with similar or the same histories? Three broad aspects need to be considered here: (1) the feature extraction method, (2) the choice of pre-processing methods for dimensionality reduction (self-organising maps, principal component analysis, kernel principal component analysis, t-distributed stochastic neighbour embedding, etc.), and (3) the clustering (k-means, hierarchical methods, DBSCAN/OPTICS) or classification methods (random forests, support vector machines, linear classifiers).

A study of geophysical image variability using a simple 2D correlation or maximal information coefficient between pairs of images of different histories would be illuminating. Do we have images which are the same as each other (or at least very similar and within the noise tolerance of the geophysical fields) but belong to very different histories? If these exist, the ambiguity of the histories can be examined, and we then know where we would expect poor performance from ML techniques which rely on easily discriminated images. The systems of equations characterising geophysical inverse problems often have a non-unique solution. In ML research, if we only use magnetic data or gravity data for inversion, we will be troubled by the non-uniqueness of the solution. However, because we have both gravity data and magnetic data, we can extract features from multi-source heterogeneous data at the same time and then classify or regress them after feature fusion. This could greatly reduce the influence of the non-unique solution. Having a large set of models will allow clustering of models according to their geophysical response and identifying subsets of geological models that are geophysically equivalent and cannot be distinguished using geophysical data. The analysis of diversity of such subsets of models will give an estimate of the severity of non-uniqueness and allow the derivation of posterior statistical indicators conditioned by geological plausibility.

In this study we have produced a ML training data set that attempts to address six recognised limitations of applying ML to geoscientific data sets, namely spatio-temporal structure; high dimensionality; small sample size; paucity of ground truth; multi-resolution data; and noise, incompleteness and uncertainty in data. Contrary to usual practice, the work for the generation of a comprehensive suite of geological models did not depend on the manual labelling of data. We relied solely on geoscientific theory and principles while remaining computationally efficient. While realistic-looking suites of geological models have been generated using generative adversarial networks (Zhang et al., 2019), these generally represent a limited range of geological scenarios and lack extensive training samples.

A DOI (10.5281/zenodo.4589883) provides access to a GitHub repository which contains the following elements (Jessell, 2021):

the source code (C language) for the version of Noddy adapted to producing random models;

This study represents our first steps in producing geologically reasonable training sets for ML and geophysical inversion applications. We have used Noddy to generate a very large, open-access 1-million-model set of 3D geology and resulting gravity and magnetic models as ML training sets. These training sets can also be used as test cases for gravity and/or magnetic inversions. The work presented here may be a first step to overcoming some of the fundamental limitations of applying these techniques to natural geoscientific data sets.