Search

Stochastic models in hydrology are very useful and widespread tools for making reliable probabilistic predictions. However, such models are only accurate at making predictions if model parameters are first of all calibrated to measured data in a consistent framework such as the Bayesian one, in which knowledge about model parameters is described through probability distributions. Unfortunately, Bayesian parameter calibration, a. k. a. inference, with stochastic models, is often a computationally intractable problem with traditional inference algorithms, such as the Metropolis algorithm, due to the expensive likelihood functions. Therefore, the prohibitive computational cost is often overcome by employing over-simplified error models, which leads to biased parameter estimates and unreliable predictions. However, thanks to recent advancements in algorithms and computing power, fully fledged Bayesian inference with stochastic models is no longer off-limits for hydrological applications. Our goal in this work is to demonstrate that a computationally efficient Hamiltonian Monte Carlo algorithm with a timescale separation makes Bayesian parameter inference with stochastic models feasible. Hydrology can potentially take great advantage of this powerful data-driven inference method as a sound calibration of model parameters is essential for making robust probabilistic predictions, which can certainly be useful in planning and policy-making. We demonstrate the Hamiltonian Monte Carlo approach by detailing a case study from urban hydrology. Discussing specific hydrological models or systems is outside the scope of our present work and will be the focus of further studies.

Conference talk, A lot of effort has gone into the development of sound physics-based stochastic solar dynamo models capable of explaining a wide variety of characteristic phenomena such as for instance periodic cycles and occurrences of Grand Minima. Building on the work of Wilmot-Smith et al. (2006), we propose a second-order stochastic delay differential equation (SDDE) model for the solar dynamo. Stochastic models can be very useful tools for making reliable probabilistic predictions. However, model parameters need to be first calibrated to measured data in a consistent framework such as the Bayesian one. Bayesian inference methods bear the great advantage over traditional optimization techniques of providing an uncertainty estimation for the calibrated parameters in the form of a probability distribution. Unfortunately, Bayesian parameter calibration with stochastic models is a computationally very challenging task, requiring sophisticated algorithms and appropriate hardware. Moreover, a reliable parameter calibration for solar dynamo models has been hindered so far by the lack of data over a time span comparable to solar dynamics time scales, as the only available direct observations of long-term solar magnetic activity are sunspot number records, covering only about 300 years. However, time-series of cosmogenic radionuclides (Brehm et al. 2021, Usoskin et al. 2021) are proxies for solar magnetic activity on a millennial time scale, which can potentially shed light on several open questions, e.g., occurrences of Grand Minima and long-period cycles, and boost our understanding and predictive capabilities of the solar dynamo. We present here our first attempts to apply advanced Bayesian inference methods to the calibration of a SDDE solar dynamo model, and discuss preliminary results, hardships, and perspectives (and dreams).

Invited talk, session "New tools for high-dimensional Bayesian inference from physics and ML", In essentially all applied sciences, data-driven modeling heavily relies on a sound calibration of model parameters to measured data for making probabilistic predictions. Bayesian statistics is a consistent framework for parameter inference where knowledge about model parameters is expressed through probability distributions. However, Bayesian inference with stochastic models can become computationally extremely expensive and it is therefore hardly ever applied. We propose a very efficient approach for boosting Bayesian parameter inference of stochastic differential equation (SDE) models calibrated to measured time-series, using a Hamiltonian Monte Carlo (HMC) approach combined with a multiple time-scale integration. We present the first application of this HMC algorithm to a real-world case study from urban hydrology.

Proxies of solar activity have revealed repeated Grand Minima that occur with a certain regularity associated with the well-known Gleissberg and Süss/deVries cycles. These and other prominent cycles in the spectrum of solar activity are also seen in the spectrum of the planetary torque exerted on the solar tachocline, which has revived the hypothesis of a planetary influence on solar activity. It is not clear, however, how the extremely weak planetary forcing could influence the solar magnetic activity. Here, we suggest that stochastic resonance could explain the necessary amplification of the forcing and provide numerical evidence from stochastic time-delayed dynamo models. If the intrinsic noise of the solar dynamo allows for a frequent switching between active and quiescent stable states, tiny periodic forcings can be greatly amplified, provided the dynamo is poised close to a critical point. Such a forcing could be caused by a tidal modulation of the minimal magnetic field required for flux-tube buoyancy.

Conceptual models are indispensable tools for hydrology. In order to use them for making probabilistic predictions, they need to be equipped with an adequate error model, which, for ease of inference, is traditionally formulated as an additive error on the output (discharge). However, the main sources of uncertainty in hydrological modelling are typically not to be found on the output, but on the input (rain) and in the model structure. Therefore, more reliable error models and probabilistic predictions can be obtained by incorporating those uncertainties directly where they arise, that is, into the model. This, however, leads us to stochastic models, which render traditional inference algorithms such as the Metropolis algorithm infeasible due to their expensive likelihood functions. However, thanks to recent advancements in algorithms and computing power, full-fledged Bayesian inference with stochastic models is no longer off-limit for hydrological applications. We demonstrate this with a case study from urban hydrology, for which we employ a highly efficient Hamiltonian Monte Carlo inference algorithm with a time-scale separation.

Approximate Bayes computations (ABC) are used for parameter inference when the likelihood function of the model is expensive to evaluate but relatively cheap to sample from. In particle ABC, an ensemble of particles in the product space of model outputs and parameters is propagated in such a way that its output marginal approaches a delta function at the data and its parameter marginal approaches the posterior distribution. Inspired by Simulated Annealing, we present a new class of particle algorithms for ABC, based on a sequence of Metropolis kernels, associated with a decreasing sequence of tolerances w.r.t. the data. Unlike other algorithms, our class of algorithms is not based on importance sampling. Hence, it does not suffer from a loss of effective sample size due to re-sampling. We prove convergence under a condition on the speed at which the tolerance is decreased. Furthermore, we present a scheme that adapts the tolerance and the jump distribution in parameter space according to some mean-fields of the ensemble, which preserves the statistical independence of the particles, in the limit of infinite sample size. This adaptive scheme aims at converging as close as possible to the correct result with as few system updates as possible via minimizing the entropy production of the process. The performance of this new class of algorithms is compared against two other recent algorithms on two toy examples as well as on a real-world example from genetics.

Oral presentation, Proxies of solar activity have revealed repeated Grand Minima that occur with a certain regularity associated with the well-known Gleissberg and Süss/deVries cycles. These and other prominent cycles in the spectrum of solar activity are also seen in the spectrum of the planetary torque exerted on the solar tachocline, which has revived the hypothesis of a planetary influence on solar activity. It is not clear, however, how the extremely weak planetary forcing could influence the solar magnetic activity. Here, we suggest that stochastic resonance could explain the necessary amplification of the forcing and provide numerical evidence from stochastic time-delayed dynamo models. If the intrinsic noise of the solar dynamo allows for a frequent switching between active and quiescent stable states, tiny periodic forcings can be greatly amplified, provided the dynamo is poised close to a critical point. Such a forcing could be caused by a tidal modulation of the minimal magnetic field required for flux-tube buoyancy.

Repeated Grand Minima revealed by proxies of solar activity suggest that the Sun alternates between two stable states, a quiescent and an active one. If the intrinsic noise of the solar dynamo allows for a frequent switching between these stable states, a tiny periodic modulation of the corresponding transition probabilities can be greatly amplified – a phenomenon known as stochastic resonance. It is well-known that Babcock-Leighton-type dynamo models can be reduced to a stochastic iterative map model capable of capturing the essential low-frequency features of the solar dynamo mechanism. In such a simplified framework, we give numerical evidence that a tiny tidal modulation of the minimal magnetic field required for flux-tube buoyancy is greatly amplified by the dynamo, provided that it operates close enough to a critical bifurcation point. Inference with more refined dynamo models is required to test this “criticality hypothesis”.

Parameter inference is a fundamental problem in data-driven modeling. Given observed data that is believed to be a realization of some parameterized model, the aim is to find parameter values that are able to explain the observed data. In many situations, the dominant sources of uncertainty must be included into the model, for making reliable predictions. This naturally leads to stochastic models. Stochastic models render parameter inference much harder, as the aim then is to find a distribution of likely parameter values. In Bayesian statistics, which is a consistent framework for data-driven learning, this so-called posterior distribution can be used to make probabilistic predictions. We propose a novel, exact and very efficient approach for generating posterior parameter distributions, for stochastic differential equation models calibrated to measured time-series. The algorithm is inspired by re-interpreting the posterior distribution as a statistical mechanics partition function of an object akin to a polymer, where the measurements are mapped on heavier beads compared to those of the simulated data. To arrive at distribution samples, we employ a Hamiltonian Monte Carlo approach combined with a multiple time-scale integration. A separation of time scales naturally arises if either the number of measurement points or the number of simulation points becomes large. Furthermore, at least for 1D problems, we can decouple the harmonic modes between measurement points and solve the fastest part of their dynamics analytically. Our approach is applicable to a wide range of inference problems and is highly parallelizable.

In essentially all applied sciences, data-driven modeling heavily relies on a sound calibration of model parameters to measured data for making probabilistic predictions. Bayesian statistics is a consistent framework for parameter inference where knowledge about model parameters is expressed through probability distributions and updated using measured data. However, Bayesian inference with non-trivial stochastic models can become computationally extremely expensive and it is therefore hardly ever applied. In recent years, sophisticated and scalable algorithms have emerged, which have the potential of making Bayesian inference for complex stochastic models feasible, even for very large data sets. We present here the power of both Approximate Bayesian Computation (ABC) and Hamiltonian Monte Carlo (HMC) algorithms through a case study in solar physics. Time-series of cosmogenic radionuclides in wood and polar ice cores are a proxy for solar activity on multi-millennial time-scales and exhibit a number of interesting and mostly not-yet-understood features such as stable cycles, Grand Minima and intermittency. Solar physicists have put a lot of effort into the development of stochastic solar dynamo models, which need to be calibrated to the observations. Parameter inference for stochastic dynamo models on long time-series of radionuclides is an open and highly topical question in solar physics. Achieving more reliable predictions of solar activity has important implications for the Earth’s climate.

Copyright ©2016 IEICE, When dealing with high-dimensional measurements that often show non-linear characteristics at multiple scales, a need for unbiased and robust classification and interpretation techniques has emerged. Here, we present a method for mapping high-dimensional data onto low-dimensional spaces, allowing for a fast visual interpretation of the data. Classical approaches of dimensionality reduction attempt to preserve the geometry of the data. They often fail to correctly grasp cluster structures, for instance in high-dimensional situations, where distances between data points tend to become more similar. In order to cope with this clustering problem, we propose to combine classical multi-dimensional scaling with data clustering based on self-organization processes in neural networks, where the goal is to amplify rather than preserve local cluster structures. We find that applying dimensionality reduction techniques to the output of neural network based clustering not only allows for a convenient visual inspection, but also leads to further insights into the intraand inter-cluster connectivity. We report on an implementation of the method with Rulkov-Hebbian-learning clustering and illustrate its suitability in comparison to traditional methods by means of an artificial dataset and a real world example.