Resampled ensemble Kalman inversion for Bayesian parameter estimation with sequential data

Jiangqi Wu; Linjie Wen; Jinglai Li

doi:10.3934/dcdss.2021045

Article Contents

2022, Volume 15, Issue 4: 837-850. Doi: 10.3934/dcdss.2021045

This issue Previous Article Numerical methods preserving multiple Hamiltonians for stochastic Poisson systems Next Article Highly accurate operator factorization methods for the integral fractional Laplacian and its generalization

Resampled ensemble Kalman inversion for Bayesian parameter estimation with sequential data

1.
School of Mathematical Sciences, Shanghai Jiao Tong University, 800 Dongchuan Rd, Shanghai 200240, China
2.
School of Mathematics, University of Birmingham, Edgbaston Birmingham, B15 2TT, UK

^* Corresponding author: Jinglai Li
^* Corresponding author: Jinglai Li

Received: December 31, 2020

Revised: January 31, 2021

Early access: April 2021

Published: April 2022

The work was supported by NSFC under grant number 11771289

Abstract Full Text(HTML) Figure(7) Related Papers Cited by

Abstract

Many real-world problems require to estimate parameters of interest in a Bayesian framework from data that are collected sequentially in time. Conventional methods to sample the posterior distributions, such as Markov Chain Monte Carlo methods can not efficiently deal with such problems as they do not take advantage of the sequential structure. To this end, the Ensemble Kalman inversion (EnKI), which updates the particles whenever a new collection of data arrive, becomes a popular tool to solve this type of problems. In this work we present a method to improve the performance of EnKI, which removes some particles that significantly deviate from the posterior distribution via a resampling procedure. Specifically we adopt an idea developed in the sequential Monte Carlo sampler, and simplify it to compute an approximate weight function. Finally we use the computed weights to identify and remove those particles seriously deviating from the target distribution. With numerical examples, we demonstrate that, without requiring any additional evaluations of the forward model, the proposed method can improve the performance of standard EnKI in certain class of problems.

Citation:

Full Text(HTML)

Figure 1. The simulated data for $ \sigma = 0.4 $ (left) and $ \sigma = 0.6 $ (right). The lines show the simulated states in continuous time and the dots are the noisy observations

Download: Full-size image PowerPoint slide

Figure 2. The results for the case where noise variance is $ 0.4^2 $. Left: the error for the simulation with $ 100 $ particles. Right: the error for simulation with $ 500 $ particles

Download: Full-size image PowerPoint slide

Figure 3. The results for the case where noise variance is $ 0.6^2 $. Left: the error for the simulation with $ 100 $ particles. Right: the error for simulation with $ 500 $ particles

Download: Full-size image PowerPoint slide

Figure 4. $ \Delta = 0.05. $ Top: the average estimation error of the simulation with 2000 particles. Bottom: the average estimation error of the simulation with 5000 particles. In both rows, the left figure shows the results of the observed dimensions and the right one shows the unobserved ones

Download: Full-size image PowerPoint slide

Figure 5. $ \Delta = 0.1. $ Top: the average estimation error of the simulation with 2000 particles. Bottom: the average estimation error of the simulation with 5000 particles. In both rows, the left figure shows the results of the observed dimensions and the right one shows the unobserved ones

Download: Full-size image PowerPoint slide

Figure 6. Left: The ground truth for x. Right: Both the noise-free and the noisy data at t = 3

Download: Full-size image PowerPoint slide

Figure 7. The estimation error for the high dimensional nonlinear example. Left: the results for $ M = 2000 $; Right: the results for $ M = 5000 $

Download: Full-size image PowerPoint slide

Related Papers

Cited by

References

[1]	J. D. Annan and J. C. Hargreaves, Efficient parameter estimation for a highly chaotic system, Tellus A: Dynamic Meteorology and Oceanography, 56 (2004), 520-526. doi: 10.3402/tellusa.v56i5.14438.
[2]	J. D. Annan, D. J. Lunt, J. C. Hargreaves and P. J. Valdes, Parameter estimation in an atmospheric gcm using the ensemble kalman filter, Nonlinear Processes in Geophysics, European Geosciences Union (EGU), 12 (2005), 363-371. doi: 10.5194/npg-12-363-2005.
[3]	A. Apte, M. Hairer, A. M. Stuart and J. Voss, Sampling the posterior: An approach to non-gaussian data assimilation, Physica D: Nonlinear Phenomena, 230 (2007), 50-64. doi: 10.1016/j.physd.2006.06.009.
[4]	A. Arnold, D. Calvetti and E. Somersalo, Parameter estimation for stiff deterministic dynamical systems via ensemble kalman filter, Inverse Problems, 30 (2014), 105008, 30pp. doi: 10.1088/0266-5611/30/10/105008.
[5]	M. S. Arulampalam, S. Maskell, N. Gordon and T. Clapp, A tutorial on particle filters for online nonlinear/non-gaussian bayesian tracking, IEEE Transactions on Signal Processing, 50 (2002), 174-188. doi: 10.1109/78.978374.
[6]	P. Del Moral, A. Doucet and A. Jasra, Sequential monte carlo samplers, Journal of the Royal Statistical Society: Series B (Statistical Methodology), 68 (2006), 411-436. doi: 10.1111/j.1467-9868.2006.00553.x.
[7]	A. Doucet and A. M. Johansen, A tutorial on particle filtering and smoothing: Fifteen years later, Handbook of Nonlinear Filtering, (2011), 656–704.
[8]	O. G. Ernst, B. Sprungk and H.-J. Starkloff, Analysis of the ensemble and polynomial chaos kalman filters in bayesian inverse problems, SIAM/ASA Journal on Uncertainty Quantification, 3 (2015), 823-851. doi: 10.1137/140981319.
[9]	G. Evensen, Data Assimilation: The Ensemble Kalman Filter, Springer Science & Business Media, 2009. doi: 10.1007/978-3-642-03711-5.
[10]	M. Frei and H. R. Künsch, Bridging the ensemble kalman and particle filters, Biometrika, 100 (2013), 781-800. doi: 10.1093/biomet/ast020.
[11]	A. Gelman, J. B. Carlin, H. S. Stern, D. Dunson, A. Vehtari and D. B. Rubin, Bayesian Data Analysis (3rd edition). Texts in Statistical Science Series. CRC Press, Boca Raton, FL, 2014.
[12]	W. R. Gilks, S. Richardson and D. Spiegelhalter, Markov Chain Monte Carlo in Practice, Interdisciplinary Statistics. Chapman & Hall, London, 1996.
[13]	A. Golightly and D. J. Wilkinson, Bayesian parameter inference for stochastic biochemical network models using particle Markov chain Monte Carlo, Interface Focus, 1 (2011), 807-820. doi: 10.1098/rsfs.2011.0047.
[14]	M. A. Iglesias, A regularizing iterative ensemble kalman method for pde-constrained inverse problems, Inverse Problems, 32 (2016), 025002, 45pp. doi: 10.1088/0266-5611/32/2/025002.
[15]	M. A Iglesias, K. J. H. Law and A. M. Stuart, Ensemble kalman methods for inverse problems, Inverse Problems, 29 (2013), 045001, 20pp. doi: 10.1088/0266-5611/29/4/045001.
[16]	E. N. Lorenz and K. A. Emanuel, Optimal sites for supplementary weather observations: Simulation with a small model, Journal of the Atmospheric Sciences, 55 (1998), 399-414. doi: 10.1175/1520-0469(1998)055<0399:OSFSWO>2.0.CO;2.
[17]	N. Papadakis, É. Mémin, A. Cuzol and N. Gengembre, Data assimilation with the weighted ensemble kalman filter, Tellus A: Dynamic Meteorology and Oceanography, 62 (2010), 673-697.
[18]	C. Schillings and A. M. Stuart, Analysis of the ensemble kalman filter for inverse problems, SIAM Journal on Numerical Analysis, 55 (2017), 1264-1290. doi: 10.1137/16M105959X.
[19]	X. Sun, L. Jin and M. Xiong, Extended kalman filter for estimation of parameters in nonlinear state-space models of biochemical networks, PloS One, 3 (2008), e3758. doi: 10.1371/journal.pone.0003758.