# American Institute of Mathematical Sciences

doi: 10.3934/fods.2020018

## Ensemble Kalman Inversion for nonlinear problems: Weights, consistency, and variance bounds

 1 Department of Mathematics, University of Wisconsin-Madison, Madison, WI 53706 USA 2 Department of Mathematics, Duke University, Durham, NC 27708 USA

* Corresponding author: Zhiyan Ding

Zhiyan Ding and Qin Li are supported in part by NSF CAREER DMS-1750488, NSF TRIPODS 1740707 and Wisconsin Data Science Initiative. The work of Jianfeng Lu is supported in part by National Science Foundation via grants DMS-1454939 and DMS-2012286. All three authors thank the two anonymous referees for the very helpful suggestions

Received  May 2020 Revised  July 2020 Published  November 2020

Ensemble Kalman Inversion (EnKI) [23] and Ensemble Square Root Filter (EnSRF) [36] are popular sampling methods for obtaining a target posterior distribution. They can be seem as one step (the analysis step) in the data assimilation method Ensemble Kalman Filter [17,3]. Despite their popularity, they are, however, not unbiased when the forward map is nonlinear [12,16,25]. Important Sampling (IS), on the other hand, obtains the unbiased sampling at the expense of large variance of weights, leading to slow convergence of high moments.

We propose WEnKI and WEnSRF, the weighted versions of EnKI and EnSRF in this paper. It follows the same gradient flow as that of EnKI/EnSRF with weight corrections. Compared to the classical methods, the new methods are unbiased, and compared with IS, the method has bounded weight variance. Both properties will be proved rigorously in this paper. We further discuss the stability of the underlying Fokker-Planck equation. This partially explains why EnKI, despite being inconsistent, performs well occasionally in nonlinear settings. Numerical evidence will be demonstrated at the end.

Citation: Zhiyan Ding, Qin Li, Jianfeng Lu. Ensemble Kalman Inversion for nonlinear problems: Weights, consistency, and variance bounds. Foundations of Data Science, doi: 10.3934/fods.2020018
##### References:

show all references

##### References:
Example $1$: from left top to bottom right: WEnKI; WEnSRF; WEnKF, as shown in Remark 1 and equation (44); IS; EnKI and EnSRF. (All evolutional equation take $\Delta t = 10^{-3}$.)
Example $2$: from left top to bottom right: WEnKI; WEnSRF; WEnKF; IS; EnKI and EnSRF
Example $3$: from left top to bottom right: WEnKI; WEnSRF; WEnKF; IS; EnKI and EnSRF
Example 3: $\log( {\rm{Var}}(Nw(t))+1)$ for WEnKI, WEnSRF and IS
Example $4$: from left top to bottom right: WEnKI; WEnSRF; WEnKF; IS; EnKI and EnSRF
Example $5$: from left top to bottom right: WEnKI; WEnSRF; WEnKF; IS; EnKI and EnSRF
Example 5: $\log( {\rm{Var}}(Nw(t))+1)$ for WEnKI, WEnSRF and IS
Error of moments estimation in Example 3
 WEnKI WEnSRF Moments Est. Re. Error Est. Re. Error $\mathbb{E}|u|^1=3.84$ 3.82 0.0056 3.88 0.0098 $\mathbb{E}|u|^2=14.90$ 14.73 0.0114 15.19 0.0192 $\mathbb{E}|u|^3=58.22$ 57.19 0.0177 59.86 0.0281 $\mathbb{E}|u|^4=229.36$ 223.79 0.0243 237.75 0.0366 $\mathbb{E}|u|^5=911.22$ 882.83 0.0312 951.95 0.0447 EnKI EnSRF Moments Est. Re. Error Est. Re. Error $\mathbb{E}|u|^1=3.84$ 3.69 0.0413 3.70 0.0391 $\mathbb{E}|u|^2=14.90$ 13.66 0.0833 13.73 0.0785 $\mathbb{E}|u|^3=58.22$ 50.90 0.1258 51.35 0.1181 $\mathbb{E}|u|^4=229.36$ 190.68 0.1687 193.24 0.1575 $\mathbb{E}|u|^5=911.22$ 718.31 0.2117 732.17 0.1965 WEnKF IS Moments Est. Re. Error Est. Re. Error $\mathbb{E}|u|^1=3.84$ 3.40 0.1156 3.52 0.0858 $\mathbb{E}|u|^2=14.90$ 11.65 0.2181 12.37 0.1699 $\mathbb{E}|u|^3=58.22$ 40.22 0.3093 43.57 0.2517 $\mathbb{E}|u|^4=229.36$ 139.72 0.3908 153.56 0.3305 $\mathbb{E}|u|^5=911.22$ 488.51 0.4639 541.71 0.4055
 WEnKI WEnSRF Moments Est. Re. Error Est. Re. Error $\mathbb{E}|u|^1=3.84$ 3.82 0.0056 3.88 0.0098 $\mathbb{E}|u|^2=14.90$ 14.73 0.0114 15.19 0.0192 $\mathbb{E}|u|^3=58.22$ 57.19 0.0177 59.86 0.0281 $\mathbb{E}|u|^4=229.36$ 223.79 0.0243 237.75 0.0366 $\mathbb{E}|u|^5=911.22$ 882.83 0.0312 951.95 0.0447 EnKI EnSRF Moments Est. Re. Error Est. Re. Error $\mathbb{E}|u|^1=3.84$ 3.69 0.0413 3.70 0.0391 $\mathbb{E}|u|^2=14.90$ 13.66 0.0833 13.73 0.0785 $\mathbb{E}|u|^3=58.22$ 50.90 0.1258 51.35 0.1181 $\mathbb{E}|u|^4=229.36$ 190.68 0.1687 193.24 0.1575 $\mathbb{E}|u|^5=911.22$ 718.31 0.2117 732.17 0.1965 WEnKF IS Moments Est. Re. Error Est. Re. Error $\mathbb{E}|u|^1=3.84$ 3.40 0.1156 3.52 0.0858 $\mathbb{E}|u|^2=14.90$ 11.65 0.2181 12.37 0.1699 $\mathbb{E}|u|^3=58.22$ 40.22 0.3093 43.57 0.2517 $\mathbb{E}|u|^4=229.36$ 139.72 0.3908 153.56 0.3305 $\mathbb{E}|u|^5=911.22$ 488.51 0.4639 541.71 0.4055
Simulation time in Example 1-3
 Case WEnKI WEnSRF EnKI EnSRF Example 1 0.362s 0.197s 0.138s 0.178s Example 2 50.041s 41.739s 26.564s 18.518s Example 3 0.198s 0.115s 0.120s 0.072s
 Case WEnKI WEnSRF EnKI EnSRF Example 1 0.362s 0.197s 0.138s 0.178s Example 2 50.041s 41.739s 26.564s 18.518s Example 3 0.198s 0.115s 0.120s 0.072s
Error of moments estimation in Example 5
 WEnKI WEnSRF Moments Est. Re. Error Est. Re. Error $\mathbb{E}|u|^1=3.32$ 3.30 0.0055 3.32 0.0017 $\mathbb{E}|u|^2=11.16$ 10.99 0.0147 11.19 0.0023 $\mathbb{E}|u|^3=38.05$ 36.99 0.0279 38.12 0.0019 $\mathbb{E}|u|^4=131.45$ 125.53 0.0451 131.47 0.0001 $\mathbb{E}|u|^5=460.56$ 429.99 0.0664 459.16 0.0030 EnKI EnSRF Moments Est. Re. Error Est. Re. Error $\mathbb{E}|u|^1=3.32$ 2.96 0.1084 3.28 0.0112 $\mathbb{E}|u|^2=11.16$ 9.07 0.1872 11.04 0.0111 $\mathbb{E}|u|^3=38.05$ 29.17 0.2332 38.25 0.0053 $\mathbb{E}|u|^4=131.45$ 100.32 0.2369 137.43 0.0455 $\mathbb{E}|u|^5=460.56$ 379.73 0.1755 516.22 0.1208 WEnKF IS Moments Est. Re. Error Est. Re. Error $\mathbb{E}|u|^1=3.32$ 3.40 0.1658 3.24 0.0245 $\mathbb{E}|u|^2=11.16$ 7.72 0.3077 10.50 0.0592 $\mathbb{E}|u|^3=38.05$ 21.74 0.4287 34.10 0.1037 $\mathbb{E}|u|^4=131.45$ 61.62 0.5313 110.81 0.1571 $\mathbb{E}|u|^5=460.56$ 175.99 0.6179 360.27 0.2178
 WEnKI WEnSRF Moments Est. Re. Error Est. Re. Error $\mathbb{E}|u|^1=3.32$ 3.30 0.0055 3.32 0.0017 $\mathbb{E}|u|^2=11.16$ 10.99 0.0147 11.19 0.0023 $\mathbb{E}|u|^3=38.05$ 36.99 0.0279 38.12 0.0019 $\mathbb{E}|u|^4=131.45$ 125.53 0.0451 131.47 0.0001 $\mathbb{E}|u|^5=460.56$ 429.99 0.0664 459.16 0.0030 EnKI EnSRF Moments Est. Re. Error Est. Re. Error $\mathbb{E}|u|^1=3.32$ 2.96 0.1084 3.28 0.0112 $\mathbb{E}|u|^2=11.16$ 9.07 0.1872 11.04 0.0111 $\mathbb{E}|u|^3=38.05$ 29.17 0.2332 38.25 0.0053 $\mathbb{E}|u|^4=131.45$ 100.32 0.2369 137.43 0.0455 $\mathbb{E}|u|^5=460.56$ 379.73 0.1755 516.22 0.1208 WEnKF IS Moments Est. Re. Error Est. Re. Error $\mathbb{E}|u|^1=3.32$ 3.40 0.1658 3.24 0.0245 $\mathbb{E}|u|^2=11.16$ 7.72 0.3077 10.50 0.0592 $\mathbb{E}|u|^3=38.05$ 21.74 0.4287 34.10 0.1037 $\mathbb{E}|u|^4=131.45$ 61.62 0.5313 110.81 0.1571 $\mathbb{E}|u|^5=460.56$ 175.99 0.6179 360.27 0.2178
 [1] Håkon Hoel, Gaukhar Shaimerdenova, Raúl Tempone. Multilevel Ensemble Kalman Filtering based on a sample average of independent EnKF estimators. Foundations of Data Science, 2020, 2 (4) : 351-390. doi: 10.3934/fods.2020017 [2] Geir Evensen, Javier Amezcua, Marc Bocquet, Alberto Carrassi, Alban Farchi, Alison Fowler, Pieter L. Houtekamer, Christopher K. Jones, Rafael J. de Moraes, Manuel Pulido, Christian Sampson, Femke C. Vossepoel. An international initiative of predicting the SARS-CoV-2 pandemic using ensemble data assimilation. Foundations of Data Science, 2020  doi: 10.3934/fods.2021001 [3] Qing-Hu Hou, Yarong Wei. Telescoping method, summation formulas, and inversion pairs. Electronic Research Archive, , () : -. doi: 10.3934/era.2021007 [4] Bixiang Wang. Mean-square random invariant manifolds for stochastic differential equations. Discrete & Continuous Dynamical Systems - A, 2021, 41 (3) : 1449-1468. doi: 10.3934/dcds.2020324 [5] Kai Yang. Scattering of the focusing energy-critical NLS with inverse square potential in the radial case. Communications on Pure & Applied Analysis, 2021, 20 (1) : 77-99. doi: 10.3934/cpaa.2020258 [6] Tingting Wu, Li Liu, Lanqiang Li, Shixin Zhu. Repeated-root constacyclic codes of length $6lp^s$. Advances in Mathematics of Communications, 2021, 15 (1) : 167-189. doi: 10.3934/amc.2020051 [7] Marc Homs-Dones. A generalization of the Babbage functional equation. Discrete & Continuous Dynamical Systems - A, 2021, 41 (2) : 899-919. doi: 10.3934/dcds.2020303 [8] Julian Tugaut. Captivity of the solution to the granular media equation. Kinetic & Related Models, , () : -. doi: 10.3934/krm.2021002 [9] Bilel Elbetch, Tounsia Benzekri, Daniel Massart, Tewfik Sari. The multi-patch logistic equation. Discrete & Continuous Dynamical Systems - B, 2021  doi: 10.3934/dcdsb.2021025 [10] Peter Poláčik, Pavol Quittner. Entire and ancient solutions of a supercritical semilinear heat equation. Discrete & Continuous Dynamical Systems - A, 2021, 41 (1) : 413-438. doi: 10.3934/dcds.2020136 [11] Jianhua Huang, Yanbin Tang, Ming Wang. Singular support of the global attractor for a damped BBM equation. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020345 [12] Stefano Bianchini, Paolo Bonicatto. Forward untangling and applications to the uniqueness problem for the continuity equation. Discrete & Continuous Dynamical Systems - A, 2020  doi: 10.3934/dcds.2020384 [13] Anh Tuan Duong, Phuong Le, Nhu Thang Nguyen. Symmetry and nonexistence results for a fractional Choquard equation with weights. Discrete & Continuous Dynamical Systems - A, 2021, 41 (2) : 489-505. doi: 10.3934/dcds.2020265 [14] Maicon Sônego. Stable transition layers in an unbalanced bistable equation. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020370 [15] François Dubois. Third order equivalent equation of lattice Boltzmann scheme. Discrete & Continuous Dynamical Systems - A, 2009, 23 (1&2) : 221-248. doi: 10.3934/dcds.2009.23.221 [16] Oleg Yu. Imanuvilov, Jean Pierre Puel. On global controllability of 2-D Burgers equation. Discrete & Continuous Dynamical Systems - A, 2009, 23 (1&2) : 299-313. doi: 10.3934/dcds.2009.23.299 [17] Biyue Chen, Chunxiang Zhao, Chengkui Zhong. The global attractor for the wave equation with nonlocal strong damping. Discrete & Continuous Dynamical Systems - B, 2021  doi: 10.3934/dcdsb.2021015 [18] Yukihiko Nakata. Existence of a period two solution of a delay differential equation. Discrete & Continuous Dynamical Systems - S, 2021, 14 (3) : 1103-1110. doi: 10.3934/dcdss.2020392 [19] Siyang Cai, Yongmei Cai, Xuerong Mao. A stochastic differential equation SIS epidemic model with regime switching. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020317 [20] Xuefei He, Kun Wang, Liwei Xu. Efficient finite difference methods for the nonlinear Helmholtz equation in Kerr medium. Electronic Research Archive, 2020, 28 (4) : 1503-1528. doi: 10.3934/era.2020079

Impact Factor: