# American Institute of Mathematical Sciences

• Previous Article
Geometry of the Kahan discretizations of planar quadratic Hamiltonian systems. Ⅱ. Systems with a linear Poisson tensor
• JCD Home
• This Issue
• Next Article
A structure-preserving Fourier pseudo-spectral linearly implicit scheme for the space-fractional nonlinear Schrödinger equation
December  2019, 6(2): 385-400. doi: 10.3934/jcd.2019019

## Discrete gradients for computational Bayesian inference

 University of Potsdam, Institute of Mathematics, Karl-Liebknecht-Str. 24/25, Potsdam, D-14476, Germany

Received  March 2019 Revised  July 2019 Published  November 2019

Fund Project: This research has been partially funded by Deutsche Forschungsgemeinschaft (DFG) through grant CRC 1294 'Data Assimilation' (projects A02 and B04).

In this paper, we exploit the gradient flow structure of continuous-time formulations of Bayesian inference in terms of their numerical time-stepping. We focus on two particular examples, namely, the continuous-time ensemble Kalman–Bucy filter and a particle discretisation of the Fokker–Planck equation associated to Brownian dynamics. Both formulations can lead to stiff differential equations which require special numerical methods for their efficient numerical implementation. We compare discrete gradient methods to alternative semi-implicit and other iterative implementations of the underlying Bayesian inference problems.

Citation: Sahani Pathiraja, Sebastian Reich. Discrete gradients for computational Bayesian inference. Journal of Computational Dynamics, 2019, 6 (2) : 385-400. doi: 10.3934/jcd.2019019
##### References:

show all references

##### References:
Exact solution and numerical approximations to the mean $m_\tau$ and variance $\sigma_\tau$ estimated by the EnKBF using the discrete gradient (DG) method with $\theta = 1$ and the semi-implicit (SI) Euler method with step-sizes (a) $\Delta \tau = 0.1$, (b) $\Delta \tau = 0.2$, (c) $\Delta \tau = 0.5$, (d) $\Delta \tau = 1.0$
Exact final values and numerical approximations to the mean $m_\tau$ and variance $\sigma_\tau$ estimated by the EnKBF using the discrete gradient (DG) with $\theta = 1$, the semi-implicit (SI) Euler, and the gradient-free, explicit (IEnKF) method with step-sizes (a) $\Delta \tau = 0.01$, (b) $\Delta \tau = 0.1$, (c) $\Delta \tau = 0.2$, (d) $\Delta \tau = 0.5$. The reference time evolution of the mean and variance is given by the explicit Euler method with step-size $\Delta \tau = 0.00025$
Exact final values and numerical approximations to the mean $m_t$ and variance $\sigma_t$ for the linear problem. Posterior is estimated by the Fokker-Planck dynamics with $M = 10$ particles using the discrete gradient (DG) method with $\theta = 1$ and the semi-implicit (SI) Euler method with step-sizes (a) $\Delta \tau = 0.004$, (b) $\Delta \tau = 0.01$, (c) $\Delta \tau = 0.04$, (d) $\Delta \tau = 0.1$. The reference time evolution of the mean and variance is given by the Explicit Euler (EE) method with step-size $\Delta \tau = 2 \times 10^{-4}$
Exact final values and numerical approximations to the mean $m_t$ and variance $\sigma_t$ for the non-linear problem. Posterior is estimated by the Fokker-Planck dynamics using the discrete gradient (DG) method with $\theta = 1$ and the semi-implicit (SI) Euler method with step-sizes (a) $\Delta \tau = 0.002$, (b) $\Delta \tau = 0.005$, (c) $\Delta \tau = 0.02$, (d) $\Delta \tau = 0.05$. The reference time evolution of the mean and variance is given by the Explicit Euler (EE) method with step-size $\Delta \tau = 2.5 \times 10^{-6}$
RMSE for sequential data assimilation applied to the Lorenz-63 model. The parameter $\alpha = 1$ corresponds to a standard square root ensemble Kalman filter, while $\alpha<1$ results in a Gaussian mixture approximation to the prior and posterior distributions. Small improvements over the ensemble Kaman filter can be found for ensemble sizes $M\ge 35$
 $\alpha\backslash M$ $15$ $20$ $25$ $30$ $35$ $40$ $45$ $50$ $0.8$ $2.5145$ $2.4726$ $2.4471$ $2.4406$ $2.4221$ $2.4173$ $2.4162$ $2.4112$ $0.85$ $2.5048$ $2.4585$ $2.4357$ $2.4330$ $2.4186$ $2.4161$ $2.4020$ $2.4026$ $0.9$ $2.4989$ $2.4493$ $2.4396$ $2.4259$ $2.4177$ $2.4103$ $2.4106$ $2.4075$ $0.95$ $2.4998$ $2.4605$ $2.4399$ $2.4327$ $2.4242$ $2.4132$ $2.4162$ $2.4131$ $1.0$ $2.4804$ $2.4597$ $2.4518$ $2.4354$ $2.4334$ $2.4247$ $2.4244$ $2.4203$
 $\alpha\backslash M$ $15$ $20$ $25$ $30$ $35$ $40$ $45$ $50$ $0.8$ $2.5145$ $2.4726$ $2.4471$ $2.4406$ $2.4221$ $2.4173$ $2.4162$ $2.4112$ $0.85$ $2.5048$ $2.4585$ $2.4357$ $2.4330$ $2.4186$ $2.4161$ $2.4020$ $2.4026$ $0.9$ $2.4989$ $2.4493$ $2.4396$ $2.4259$ $2.4177$ $2.4103$ $2.4106$ $2.4075$ $0.95$ $2.4998$ $2.4605$ $2.4399$ $2.4327$ $2.4242$ $2.4132$ $2.4162$ $2.4131$ $1.0$ $2.4804$ $2.4597$ $2.4518$ $2.4354$ $2.4334$ $2.4247$ $2.4244$ $2.4203$
 [1] Guillaume Bal, Ian Langmore, Youssef Marzouk. Bayesian inverse problems with Monte Carlo forward models. Inverse Problems & Imaging, 2013, 7 (1) : 81-105. doi: 10.3934/ipi.2013.7.81 [2] Theodore Papamarkou, Alexey Lindo, Eric B. Ford. Geometric adaptive Monte Carlo in random environment. Foundations of Data Science, 2021, 3 (2) : 201-224. doi: 10.3934/fods.2021014 [3] Giacomo Dimarco. The moment guided Monte Carlo method for the Boltzmann equation. Kinetic & Related Models, 2013, 6 (2) : 291-315. doi: 10.3934/krm.2013.6.291 [4] Ajay Jasra, Kody J. H. Law, Yaxian Xu. Markov chain simulation for multilevel Monte Carlo. Foundations of Data Science, 2021, 3 (1) : 27-47. doi: 10.3934/fods.2021004 [5] Deng Lu, Maria De Iorio, Ajay Jasra, Gary L. Rosner. Bayesian inference for latent chain graphs. Foundations of Data Science, 2020, 2 (1) : 35-54. doi: 10.3934/fods.2020003 [6] Marc Bocquet, Julien Brajard, Alberto Carrassi, Laurent Bertino. Bayesian inference of chaotic dynamics by merging data assimilation, machine learning and expectation-maximization. Foundations of Data Science, 2020, 2 (1) : 55-80. doi: 10.3934/fods.2020004 [7] Jin-Won Kim, Amirhossein Taghvaei, Yongxin Chen, Prashant G. Mehta. Feedback particle filter for collective inference. Foundations of Data Science, 2021, 3 (3) : 543-561. doi: 10.3934/fods.2021018 [8] Evangelos Evangelou. Approximate Bayesian inference for geostatistical generalised linear models. Foundations of Data Science, 2019, 1 (1) : 39-60. doi: 10.3934/fods.2019002 [9] Michael B. Giles, Kristian Debrabant, Andreas Rössler. Analysis of multilevel Monte Carlo path simulation using the Milstein discretisation. Discrete & Continuous Dynamical Systems - B, 2019, 24 (8) : 3881-3903. doi: 10.3934/dcdsb.2018335 [10] Jiakou Wang, Margaret J. Slattery, Meghan Henty Hoskins, Shile Liang, Cheng Dong, Qiang Du. Monte carlo simulation of heterotypic cell aggregation in nonlinear shear flow. Mathematical Biosciences & Engineering, 2006, 3 (4) : 683-696. doi: 10.3934/mbe.2006.3.683 [11] Domokos Szász. Algebro-geometric methods for hard ball systems. Discrete & Continuous Dynamical Systems, 2008, 22 (1&2) : 427-443. doi: 10.3934/dcds.2008.22.427 [12] Yanyan Shi, Yajuan Sun, Yulei Wang, Jian Liu. Study of adaptive symplectic methods for simulating charged particle dynamics. Journal of Computational Dynamics, 2019, 6 (2) : 429-448. doi: 10.3934/jcd.2019022 [13] Chjan C. Lim, Joseph Nebus, Syed M. Assad. Monte-Carlo and polyhedron-based simulations I: extremal states of the logarithmic N-body problem on a sphere. Discrete & Continuous Dynamical Systems - B, 2003, 3 (3) : 313-342. doi: 10.3934/dcdsb.2003.3.313 [14] Joseph Nebus. The Dirichlet quotient of point vortex interactions on the surface of the sphere examined by Monte Carlo experiments. Discrete & Continuous Dynamical Systems - B, 2005, 5 (1) : 125-136. doi: 10.3934/dcdsb.2005.5.125 [15] Olli-Pekka Tossavainen, Daniel B. Work. Markov Chain Monte Carlo based inverse modeling of traffic flows using GPS data. Networks & Heterogeneous Media, 2013, 8 (3) : 803-824. doi: 10.3934/nhm.2013.8.803 [16] Mazyar Zahedi-Seresht, Gholam-Reza Jahanshahloo, Josef Jablonsky, Sedighe Asghariniya. A new Monte Carlo based procedure for complete ranking efficient units in DEA models. Numerical Algebra, Control & Optimization, 2017, 7 (4) : 403-416. doi: 10.3934/naco.2017025 [17] Nicolas Forcadel, Cyril Imbert, Régis Monneau. Homogenization of some particle systems with two-body interactions and of the dislocation dynamics. Discrete & Continuous Dynamical Systems, 2009, 23 (3) : 785-826. doi: 10.3934/dcds.2009.23.785 [18] Doron Levy, Tiago Requeijo. Modeling group dynamics of phototaxis: From particle systems to PDEs. Discrete & Continuous Dynamical Systems - B, 2008, 9 (1) : 103-128. doi: 10.3934/dcdsb.2008.9.103 [19] Luca Dieci, Cinzia Elia. Smooth to discontinuous systems: A geometric and numerical method for slow-fast dynamics. Discrete & Continuous Dynamical Systems - B, 2018, 23 (7) : 2935-2950. doi: 10.3934/dcdsb.2018112 [20] Oliver J. Maclaren, Helen M. Byrne, Alexander G. Fletcher, Philip K. Maini. Models, measurement and inference in epithelial tissue dynamics. Mathematical Biosciences & Engineering, 2015, 12 (6) : 1321-1340. doi: 10.3934/mbe.2015.12.1321

Impact Factor: