
-
Previous Article
Geometry of the Kahan discretizations of planar quadratic Hamiltonian systems. Ⅱ. Systems with a linear Poisson tensor
- JCD Home
- This Issue
-
Next Article
A structure-preserving Fourier pseudo-spectral linearly implicit scheme for the space-fractional nonlinear Schrödinger equation
Discrete gradients for computational Bayesian inference
University of Potsdam, Institute of Mathematics, Karl-Liebknecht-Str. 24/25, Potsdam, D-14476, Germany |
In this paper, we exploit the gradient flow structure of continuous-time formulations of Bayesian inference in terms of their numerical time-stepping. We focus on two particular examples, namely, the continuous-time ensemble Kalman–Bucy filter and a particle discretisation of the Fokker–Planck equation associated to Brownian dynamics. Both formulations can lead to stiff differential equations which require special numerical methods for their efficient numerical implementation. We compare discrete gradient methods to alternative semi-implicit and other iterative implementations of the underlying Bayesian inference problems.
References:
[1] |
J. Amezcua, E. Kalnay, K. Ide and S. Reich,
Ensemble transform Kalman-Bucy filters, Q. J. R. Meteor. Soc., 140 (2014), 995-1004.
doi: 10.1002/qj.2186. |
[2] |
U. M. Ascher, Numerical Methods for Evolutionary Differential Equations, Computational Science & Engineering, 5. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 2008.
doi: 10.1137/1.9780898718911. |
[3] |
K. Bergemann and S. Reich,
A localization technique for ensemble Kalman filters, Q. J. R. Meteorological Soc., 136 (2010), 701-707.
doi: 10.1002/qj.591. |
[4] |
K. Bergemann and S. Reich,
A mollified ensemble Kalman filter, Q. J. R. Meteorological Soc., 136 (2010), 1636-1643.
doi: 10.1002/qj.672. |
[5] |
K. Bergemann and S. Reich,
An ensemble Kalman-Bucy filter for continuous data assimilation, Meteorolog. Zeitschrift, 21 (2012), 213-219.
doi: 10.1127/0941-2948/2012/0307. |
[6] |
D. Blömker, C. Schillings and P. Wacker,
A strongly convergent numerical scheme for ensemble Kalman inversion, SIAM J. Numer. Anal., 56 (2018), 2537-2562.
doi: 10.1137/17M1132367. |
[7] |
Y. Chen and D. S. Oliver,
Levenberg-Marquardt forms of the iterative ensemble smoother for efficient history matching and uncertainty quantification, Computational Geoscience, 17 (2013), 689-703.
doi: 10.1007/s10596-013-9351-5. |
[8] |
N. Chustagulprom, S. Reich and M. Reinhardt,
A hybrid ensemble transform filter for nonlinear and spatially extended dynamical systems, SIAM/ASA J. Uncertainty Quantification, 4 (2016), 592-608.
doi: 10.1137/15M1040967. |
[9] |
D. Crisan and J. Xiong,
Approximate McKean-Vlasov representation for a class of SPDEs, Stochastics, 82 (2010), 53-68.
doi: 10.1080/17442500902723575. |
[10] |
F. Daum and J. Huang, Particle filter for nonlinear filters, in Acoustics, Speech and Signal Processing (ICASSP), 2011 IEEE International Conference on, (2011), 5920–5923. Google Scholar |
[11] |
J. de Wiljes, S. Reich and W. Stannat,
Long-time stability and accuracy of the ensemble Kalman-Bucy filter for fully observed processes and small measurement noise, SIAM J. Appl. Dyn. Syst., 17 (2018), 1152-1181.
doi: 10.1137/17M1119056. |
[12] |
P. Degond and F.-J. Mustieles,
A deterministic approximation of diffusion equations using particles, SIAM J. Sci. Comput., 11 (1990), 293-310.
doi: 10.1137/0911018. |
[13] |
G. Detommaso, T. Cui, A. Spantini, Y. Marzouk and R. Scheichl, A Stein variational Newton method, Advances in Neural Information Processing Systems (NIPS 2018), 31 (2018), 9187-9197. Google Scholar |
[14] |
A. A. Emerik and A. C. Reynolds,
Ensemble smoother with multiple data assimilation, Computers & Geosciences, 55 (2013), 3-15.
doi: 10.1016/j.cageo.2012.03.011. |
[15] |
G. Evensen, Data Assimilation. The Ensemble Kalman Filter, Second edition, Springer-Verlag, Berlin, 2009.
doi: 10.1007/978-3-642-03711-5. |
[16] |
A. Garbuno-Inigo, F. Hoffmann, W. Li and A. Stuart, Gradient Structure for the Ensemble Kalman Flow with Noise, Technical Report arXiv: 1903.08866.v2, Caltech, 2019. Google Scholar |
[17] |
O. Gonzalez,
Time integration of discrete Hamiltonian systems, J. Nonlinear Sci., 6 (1996), 449-467.
doi: 10.1007/BF02440162. |
[18] |
J. Kaipio and E. Somersalo, Statistical and Computational Inverse Problems, Applied Mathematical Sciences, 160. Springer-Verlag, New York, 2005. |
[19] |
N. B. Kovachki and A. M. Stuart, Ensemble Kalman inversion: A derivative-free technique for machine learning tasks, Inverse Problems, 35 (2019), 095005, 35 pp, arXiv: 1808.03620. |
[20] |
K. Law, A. Stuart and K. Zygalakis, Data Assimilation: A Mathematical Introduction, Texts in Applied Mathematics, 62. Springer, Cham, 2015.
doi: 10.1007/978-3-319-20325-6. |
[21] |
Q. Liu and D. Wang, Stein variational gradient descent: A general purpose Bayesian inference algorithm, in Advances in Neural Information Processing Systems (NIPS 2016), 29 (2016), 2378–2386. Google Scholar |
[22] |
E. Lorenz, Deterministic non-periodic flows, J. Atmos. Sci., 20 (1963), 130-141. Google Scholar |
[23] |
R. I. McLachlan, G. R. W. Quispel and N. Robidoux,
Geometric integration using discrete gradients, Phil Trans. R. Soc. Lond. A Math. Phys. Eng. Sci., 357 (1999), 1021-1045.
doi: 10.1098/rsta.1999.0363. |
[24] |
J. Nocedal and S. J. Wright, Numerical Optimization, Second edition. Springer Series in Operations Research and Financial Engineering. Springer, New York, 2006. |
[25] |
Y. Ollivier,
Online natural gradient as a Kalman filter, Electronic Journal of Statistics, 12 (2018), 2930-2961.
doi: 10.1214/18-EJS1468. |
[26] |
G. Pavliotis, Stochastic Processes and Applications. Diffusion Processes, the Fokker-Planck and Langevin Equations, Texts in Applied Mathematics, 60. Springer, New York, 2014.
doi: 10.1007/978-1-4939-1323-7. |
[27] |
S. Reich,
Enhancing energy conserving methods, BIT, 36 (1996), 122-134.
doi: 10.1007/BF01740549. |
[28] |
S. Reich,
A dynamical systems framework for intermittent data assimilation, BIT Numer Math, 51 (2011), 235-249.
doi: 10.1007/s10543-010-0302-4. |
[29] |
S. Reich,
Data assimilation: The Schrödinger perspective, Acta Numerica, 28 (2019), 635-710.
doi: 10.1017/S0962492919000011. |
[30] |
S. Reich and C. Cotter, Probabilistic Forecasting and Bayesian Data Assimilation, Cambridge University Press, Cambridge, 2015.
doi: 10.1017/CBO9781107706804.![]() ![]() |
[31] |
C. P. Robert, The Bayesian Choice: From Decision-Theoretic Motivations to Computational Implementations, Second edition, Springer Texts in Statistics, Springer-Verlag, New York, 2001. |
[32] |
G. Russo,
Deterministic diffusion of particles, Comm. Pure Appl. Math., 43 (1990), 697-733.
doi: 10.1002/cpa.3160430602. |
[33] |
P. Sakov and P. Oke, A deterministic formulation of the ensemble Kalman filter: An alternative to ensemble square root filters, Tellus, 60A (2008), 361-371. Google Scholar |
[34] |
P. Sakov, D. Oliver and L. Bertino,
An iterative EnKF for strongly nonlinear systems, Mon. Wea. Rev., 140 (2012), 1988-2004.
doi: 10.1175/MWR-D-11-00176.1. |
[35] |
A. M. Stuart, Numerical analysis and dynamical systems, Acta Numer., Cambridge Univ. Press, Cambridge, 3 (1994), 467–572.
doi: 10.1017/S0962492900002488. |
[36] |
A. Tarantola, Inverse Problem Theory and Methods for Model Parameter Estimation, Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 2005.
doi: 10.1137/1.9780898717921. |
[37] |
H. Yserentant,
A new class of particle methods, Numer. Math., 76 (1997), 87-109.
doi: 10.1007/s002110050255. |
show all references
References:
[1] |
J. Amezcua, E. Kalnay, K. Ide and S. Reich,
Ensemble transform Kalman-Bucy filters, Q. J. R. Meteor. Soc., 140 (2014), 995-1004.
doi: 10.1002/qj.2186. |
[2] |
U. M. Ascher, Numerical Methods for Evolutionary Differential Equations, Computational Science & Engineering, 5. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 2008.
doi: 10.1137/1.9780898718911. |
[3] |
K. Bergemann and S. Reich,
A localization technique for ensemble Kalman filters, Q. J. R. Meteorological Soc., 136 (2010), 701-707.
doi: 10.1002/qj.591. |
[4] |
K. Bergemann and S. Reich,
A mollified ensemble Kalman filter, Q. J. R. Meteorological Soc., 136 (2010), 1636-1643.
doi: 10.1002/qj.672. |
[5] |
K. Bergemann and S. Reich,
An ensemble Kalman-Bucy filter for continuous data assimilation, Meteorolog. Zeitschrift, 21 (2012), 213-219.
doi: 10.1127/0941-2948/2012/0307. |
[6] |
D. Blömker, C. Schillings and P. Wacker,
A strongly convergent numerical scheme for ensemble Kalman inversion, SIAM J. Numer. Anal., 56 (2018), 2537-2562.
doi: 10.1137/17M1132367. |
[7] |
Y. Chen and D. S. Oliver,
Levenberg-Marquardt forms of the iterative ensemble smoother for efficient history matching and uncertainty quantification, Computational Geoscience, 17 (2013), 689-703.
doi: 10.1007/s10596-013-9351-5. |
[8] |
N. Chustagulprom, S. Reich and M. Reinhardt,
A hybrid ensemble transform filter for nonlinear and spatially extended dynamical systems, SIAM/ASA J. Uncertainty Quantification, 4 (2016), 592-608.
doi: 10.1137/15M1040967. |
[9] |
D. Crisan and J. Xiong,
Approximate McKean-Vlasov representation for a class of SPDEs, Stochastics, 82 (2010), 53-68.
doi: 10.1080/17442500902723575. |
[10] |
F. Daum and J. Huang, Particle filter for nonlinear filters, in Acoustics, Speech and Signal Processing (ICASSP), 2011 IEEE International Conference on, (2011), 5920–5923. Google Scholar |
[11] |
J. de Wiljes, S. Reich and W. Stannat,
Long-time stability and accuracy of the ensemble Kalman-Bucy filter for fully observed processes and small measurement noise, SIAM J. Appl. Dyn. Syst., 17 (2018), 1152-1181.
doi: 10.1137/17M1119056. |
[12] |
P. Degond and F.-J. Mustieles,
A deterministic approximation of diffusion equations using particles, SIAM J. Sci. Comput., 11 (1990), 293-310.
doi: 10.1137/0911018. |
[13] |
G. Detommaso, T. Cui, A. Spantini, Y. Marzouk and R. Scheichl, A Stein variational Newton method, Advances in Neural Information Processing Systems (NIPS 2018), 31 (2018), 9187-9197. Google Scholar |
[14] |
A. A. Emerik and A. C. Reynolds,
Ensemble smoother with multiple data assimilation, Computers & Geosciences, 55 (2013), 3-15.
doi: 10.1016/j.cageo.2012.03.011. |
[15] |
G. Evensen, Data Assimilation. The Ensemble Kalman Filter, Second edition, Springer-Verlag, Berlin, 2009.
doi: 10.1007/978-3-642-03711-5. |
[16] |
A. Garbuno-Inigo, F. Hoffmann, W. Li and A. Stuart, Gradient Structure for the Ensemble Kalman Flow with Noise, Technical Report arXiv: 1903.08866.v2, Caltech, 2019. Google Scholar |
[17] |
O. Gonzalez,
Time integration of discrete Hamiltonian systems, J. Nonlinear Sci., 6 (1996), 449-467.
doi: 10.1007/BF02440162. |
[18] |
J. Kaipio and E. Somersalo, Statistical and Computational Inverse Problems, Applied Mathematical Sciences, 160. Springer-Verlag, New York, 2005. |
[19] |
N. B. Kovachki and A. M. Stuart, Ensemble Kalman inversion: A derivative-free technique for machine learning tasks, Inverse Problems, 35 (2019), 095005, 35 pp, arXiv: 1808.03620. |
[20] |
K. Law, A. Stuart and K. Zygalakis, Data Assimilation: A Mathematical Introduction, Texts in Applied Mathematics, 62. Springer, Cham, 2015.
doi: 10.1007/978-3-319-20325-6. |
[21] |
Q. Liu and D. Wang, Stein variational gradient descent: A general purpose Bayesian inference algorithm, in Advances in Neural Information Processing Systems (NIPS 2016), 29 (2016), 2378–2386. Google Scholar |
[22] |
E. Lorenz, Deterministic non-periodic flows, J. Atmos. Sci., 20 (1963), 130-141. Google Scholar |
[23] |
R. I. McLachlan, G. R. W. Quispel and N. Robidoux,
Geometric integration using discrete gradients, Phil Trans. R. Soc. Lond. A Math. Phys. Eng. Sci., 357 (1999), 1021-1045.
doi: 10.1098/rsta.1999.0363. |
[24] |
J. Nocedal and S. J. Wright, Numerical Optimization, Second edition. Springer Series in Operations Research and Financial Engineering. Springer, New York, 2006. |
[25] |
Y. Ollivier,
Online natural gradient as a Kalman filter, Electronic Journal of Statistics, 12 (2018), 2930-2961.
doi: 10.1214/18-EJS1468. |
[26] |
G. Pavliotis, Stochastic Processes and Applications. Diffusion Processes, the Fokker-Planck and Langevin Equations, Texts in Applied Mathematics, 60. Springer, New York, 2014.
doi: 10.1007/978-1-4939-1323-7. |
[27] |
S. Reich,
Enhancing energy conserving methods, BIT, 36 (1996), 122-134.
doi: 10.1007/BF01740549. |
[28] |
S. Reich,
A dynamical systems framework for intermittent data assimilation, BIT Numer Math, 51 (2011), 235-249.
doi: 10.1007/s10543-010-0302-4. |
[29] |
S. Reich,
Data assimilation: The Schrödinger perspective, Acta Numerica, 28 (2019), 635-710.
doi: 10.1017/S0962492919000011. |
[30] |
S. Reich and C. Cotter, Probabilistic Forecasting and Bayesian Data Assimilation, Cambridge University Press, Cambridge, 2015.
doi: 10.1017/CBO9781107706804.![]() ![]() |
[31] |
C. P. Robert, The Bayesian Choice: From Decision-Theoretic Motivations to Computational Implementations, Second edition, Springer Texts in Statistics, Springer-Verlag, New York, 2001. |
[32] |
G. Russo,
Deterministic diffusion of particles, Comm. Pure Appl. Math., 43 (1990), 697-733.
doi: 10.1002/cpa.3160430602. |
[33] |
P. Sakov and P. Oke, A deterministic formulation of the ensemble Kalman filter: An alternative to ensemble square root filters, Tellus, 60A (2008), 361-371. Google Scholar |
[34] |
P. Sakov, D. Oliver and L. Bertino,
An iterative EnKF for strongly nonlinear systems, Mon. Wea. Rev., 140 (2012), 1988-2004.
doi: 10.1175/MWR-D-11-00176.1. |
[35] |
A. M. Stuart, Numerical analysis and dynamical systems, Acta Numer., Cambridge Univ. Press, Cambridge, 3 (1994), 467–572.
doi: 10.1017/S0962492900002488. |
[36] |
A. Tarantola, Inverse Problem Theory and Methods for Model Parameter Estimation, Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 2005.
doi: 10.1137/1.9780898717921. |
[37] |
H. Yserentant,
A new class of particle methods, Numer. Math., 76 (1997), 87-109.
doi: 10.1007/s002110050255. |




[1] |
Qian Liu, Shuang Liu, King-Yeung Lam. Asymptotic spreading of interacting species with multiple fronts Ⅰ: A geometric optics approach. Discrete & Continuous Dynamical Systems - A, 2020, 40 (6) : 3683-3714. doi: 10.3934/dcds.2020050 |
[2] |
Predrag S. Stanimirović, Branislav Ivanov, Haifeng Ma, Dijana Mosić. A survey of gradient methods for solving nonlinear optimization. Electronic Research Archive, 2020, 28 (4) : 1573-1624. doi: 10.3934/era.2020115 |
[3] |
Gabrielle Nornberg, Delia Schiera, Boyan Sirakov. A priori estimates and multiplicity for systems of elliptic PDE with natural gradient growth. Discrete & Continuous Dynamical Systems - A, 2020, 40 (6) : 3857-3881. doi: 10.3934/dcds.2020128 |
[4] |
Shin-Ichiro Ei, Hiroshi Ishii. The motion of weakly interacting localized patterns for reaction-diffusion systems with nonlocal effect. Discrete & Continuous Dynamical Systems - B, 2021, 26 (1) : 173-190. doi: 10.3934/dcdsb.2020329 |
[5] |
Thomas Frenzel, Matthias Liero. Effective diffusion in thin structures via generalized gradient systems and EDP-convergence. Discrete & Continuous Dynamical Systems - S, 2021, 14 (1) : 395-425. doi: 10.3934/dcdss.2020345 |
[6] |
Hua Shi, Xiang Zhang, Yuyan Zhang. Complex planar Hamiltonian systems: Linearization and dynamics. Discrete & Continuous Dynamical Systems - A, 2020 doi: 10.3934/dcds.2020406 |
[7] |
Lei Yang, Lianzhang Bao. Numerical study of vanishing and spreading dynamics of chemotaxis systems with logistic source and a free boundary. Discrete & Continuous Dynamical Systems - B, 2021, 26 (2) : 1083-1109. doi: 10.3934/dcdsb.2020154 |
[8] |
Michel Chipot, Mingmin Zhang. On some model problem for the propagation of interacting species in a special environment. Discrete & Continuous Dynamical Systems - A, 2020 doi: 10.3934/dcds.2020401 |
[9] |
Philippe G. Ciarlet, Liliana Gratie, Cristinel Mardare. Intrinsic methods in elasticity: a mathematical survey. Discrete & Continuous Dynamical Systems - A, 2009, 23 (1&2) : 133-164. doi: 10.3934/dcds.2009.23.133 |
[10] |
Martin Heida, Stefan Neukamm, Mario Varga. Stochastic homogenization of $ \Lambda $-convex gradient flows. Discrete & Continuous Dynamical Systems - S, 2021, 14 (1) : 427-453. doi: 10.3934/dcdss.2020328 |
[11] |
Meng Chen, Yong Hu, Matteo Penegini. On projective threefolds of general type with small positive geometric genus. Electronic Research Archive, , () : -. doi: 10.3934/era.2020117 |
[12] |
Feifei Cheng, Ji Li. Geometric singular perturbation analysis of Degasperis-Procesi equation with distributed delay. Discrete & Continuous Dynamical Systems - A, 2021, 41 (2) : 967-985. doi: 10.3934/dcds.2020305 |
[13] |
João Vitor da Silva, Hernán Vivas. Sharp regularity for degenerate obstacle type problems: A geometric approach. Discrete & Continuous Dynamical Systems - A, 2021, 41 (3) : 1359-1385. doi: 10.3934/dcds.2020321 |
[14] |
Lekbir Afraites, Chorouk Masnaoui, Mourad Nachaoui. Shape optimization method for an inverse geometric source problem and stability at critical shape. Discrete & Continuous Dynamical Systems - S, 2021 doi: 10.3934/dcdss.2021006 |
[15] |
Xing-Bin Pan. Variational and operator methods for Maxwell-Stokes system. Discrete & Continuous Dynamical Systems - A, 2020, 40 (6) : 3909-3955. doi: 10.3934/dcds.2020036 |
[16] |
Elena Nozdrinova, Olga Pochinka. Solution of the 33rd Palis-Pugh problem for gradient-like diffeomorphisms of a two-dimensional sphere. Discrete & Continuous Dynamical Systems - A, 2021, 41 (3) : 1101-1131. doi: 10.3934/dcds.2020311 |
[17] |
Peter Frolkovič, Karol Mikula, Jooyoung Hahn, Dirk Martin, Branislav Basara. Flux balanced approximation with least-squares gradient for diffusion equation on polyhedral mesh. Discrete & Continuous Dynamical Systems - S, 2021, 14 (3) : 865-879. doi: 10.3934/dcdss.2020350 |
[18] |
Xuefei He, Kun Wang, Liwei Xu. Efficient finite difference methods for the nonlinear Helmholtz equation in Kerr medium. Electronic Research Archive, 2020, 28 (4) : 1503-1528. doi: 10.3934/era.2020079 |
[19] |
Xin Guo, Lei Shi. Preface of the special issue on analysis in data science: Methods and applications. Mathematical Foundations of Computing, 2020, 3 (4) : i-ii. doi: 10.3934/mfc.2020026 |
[20] |
Shun Zhang, Jianlin Jiang, Su Zhang, Yibing Lv, Yuzhen Guo. ADMM-type methods for generalized multi-facility Weber problem. Journal of Industrial & Management Optimization, 2020 doi: 10.3934/jimo.2020171 |
Impact Factor:
Tools
Metrics
Other articles
by authors
[Back to Top]