Article Contents
Article Contents

# ISALT: Inference-based schemes adaptive to large time-stepping for locally Lipschitz ergodic systems

• * Corresponding author: Fei Lu

XL is supported by NSF DMS CAREER-1847770. FL is supported NSF DMS 1913243 and NSF DMS 1821211. FY is supported by AMS-Simons travel grants

• Efficient simulation of SDEs is essential in many applications, particularly for ergodic systems that demand efficient simulation of both short-time dynamics and large-time statistics. However, locally Lipschitz SDEs often require special treatments such as implicit schemes with small time-steps to accurately simulate the ergodic measures. We introduce a framework to construct inference-based schemes adaptive to large time-steps (ISALT) from data, achieving a reduction in time by several orders of magnitudes. The key is the statistical learning of an approximation to the infinite-dimensional discrete-time flow map. We explore the use of numerical schemes (such as the Euler-Maruyama, the hybrid RK4, and an implicit scheme) to derive informed basis functions, leading to a parameter inference problem. We introduce a scalable algorithm to estimate the parameters by least squares, and we prove the convergence of the estimators as data size increases.

We test the ISALT on three non-globally Lipschitz SDEs: the 1D double-well potential, a 2D multiscale gradient system, and the 3D stochastic Lorenz equation with a degenerate noise. Numerical results show that ISALT can tolerate time-step magnitudes larger than plain numerical schemes. It reaches optimal accuracy in reproducing the invariant measure when the time-step is medium-large.

Mathematics Subject Classification: Primary: 65C30, 60H35; Secondary: 37M25, 62M20.

 Citation:

• Figure 1.  Schematic plot of inferring explicit scheme with a large time-step

Figure 2.  Large-time statistics for 1D double-well potential. (a) TVD between the empirical invariant densities (PDF) of the inferred schemes and the reference PDF from data. (b) and (c): PDFs and ACFs comparison between the IS-RK4 with $c_0$ excluded and the reference data

Figure 3.  1D double-well potential: Convergence of estimators in IS-RK4 with $c_0$ excluded. (a) The relative error of the estimator $\widehat{c_1^{{\delta}, N,M}}$ with ${\delta} = 80\times \Delta t$ converges at an order about $(MN)^{-1/2}$, matching Theorem 3.5. (b) Left column: The coefficients depend on the time-step ${\delta} = {\mathrm{Gap}}\times \Delta t$, with $c_1$ being almost 1 and $c_2$ being close to linear in ${\delta}$ until ${\delta}>0.08$. The error bars, which are too narrow to be seen, are the standard deviations of the single-trajectory estimators from the $M$-trajectory estimator. Right column: The residual decays at an order $O({\delta}^{1/2})$, matching Theorem 3.6

Figure 4.  Large-time statistics for the 2D gradient system. (a) TVD between the $x_1$ marginal invariant densities (PDF) of the inferred schemes and the reference PDF from data. (b) and (c): PDFs and ACFs comparison between IS-SSBE with $c_0$ excluded and the reference data

Figure 5.  2D gradient system: Convergence of estimators in IS-SSBE with $c_0$ excluded. (a) The relative error of the estimator $\widehat{c_1^{{\delta}, N,M}}$ with ${\delta} = 120 \Delta t$ converges at an order about $(MN)^{-1/2}$, matching Theorem 3.5. (b) Left column: The estimators of $c_1, c_2$ are almost linear in ${\delta}$. Right column: The residual changes little as ${\delta}$ decreases, due to that IS-SSBE is not a parametrization of an explicit scheme (thus, Theorem 3.6 does not apply)

Figure 6.  2D gradient system: Convergence of estimators in IS-RK4 with $c_0$ excluded. (a) The relative error of the estimator $\widehat{c_1^{{\delta}, N,M}}$ with ${\delta} = 120 \Delta t$ converges at an order about $(MN)^{-1/2}$, matching Theorem 3.5. (b) Left column: The estimators of $c_1, c_2$ are constant for all ${\delta}$. Right column: The residual decays at an order $O({\delta}^{1/2})$, matching Theorem 3.6

Figure 7.  Large-time statistics of $x_1$ for the stochastic Lorenz system. (a) TVD between the $x_1$ marginal invariant densities (PDF) of the inferred schemes and the reference PDF from data. (b) and (c): PDFs and ACFs comparison between IS-RK4 with $c_0$ included and the reference data

Figure 8.  ACF and PDF of $x_3$ in the stochastic Lorenz system. Similar to the other examples, IS-RK4 (with $c_0$ included) reproduces the PDF and the ACF the best when the time-step is medium large, while plain RK4 and IS-EM blow up even when ${\mathrm{Gap}} = 20$

Figure 9.  The 3D stochastic Lorenz system: Convergence of estimators in IS-RK4 with $c_0$ included. (a) The relative error of the estimator $\widehat{c_1^{{\delta}, N,M}}$ with ${\delta} = 240 \Delta t = 0.12$ converges at order about $(MN)^{-1/2}$, matching Theorem 3.5. (b) Left column: The estimators of $c_0,c_1, c_2$ are varies little until ${\delta}>0.12$. The vertical dash line is the optimal time gap. Right column: The residuals decay at orders slightly higher than $O({\delta}^{1/2})$

Table 1.  Notations

 Notation Description ${{\bf X}}_t$ and ${{\bf B}}_t$ true state process and original stochastic force $f({{\bf X}}_t)$, $\sigma\in \mathbb{R}^{d\times m}$ local-Lipschitz drift and diffusion matrix $dt$ time-step generating data ${\delta}= {\mathrm{Gap}} \times dt$ time-step for inferred scheme, ${\mathrm{Gap}}\in \{ 1, 2, 4, 10, 20, 40,\ldots\}$ $t_i = i{\delta}$ discrete time instants of data $\{{{\bf X}}_{t_0:t_N}^{(m)}, {{\bf B}}_{t_0:t_N}^{(m)}\}_{m=1}^M$ Data: $M$ independent paths of ${{\bf X}}$ and ${{\bf B}}$ at discrete-times $\mathcal{F}\left({{\bf X}}_{t_i},\, {{\bf B}}_{[t_{i}, \, t_{i+1})}\right)$ true flow map representing $({{\bf X}}_{t_{i+1}}-{{\bf X}}_{t_i})/{\delta}$ ${F}^{\delta}({{\bf X}}_{t_n},\Delta {{\bf B}}_{t_n})$ approximate flow map using only ${{\bf X}}_{t_n}$, $\Delta {{\bf B}}_{t_n} = {{\bf B}}_{t_{n+1}}-{{\bf B}}_{t_{n}}$ $\widetilde F^{\delta}\left(c^{\delta}, {{\bf X}}_{t_n},\Delta {{\bf B}}_{t_n} \right)$ parametric approximate flow map $c^{\delta}=(c_0^{\delta},\dots,c_p^{\delta})$ parameters to be estimated for the inferred scheme $\eta_n$ and $\sigma_{\eta}^{\delta}$ iid $N(0, I_d)$ and covariance, representing regression residual EM and IS-EM Euler-Maruyama and inferred scheme (IS) parametrizing it HRK4 and IS-RK4 hybrid RK4 and inferred scheme parametrizing RK4 SSBE and IS-SSBE split-step stochastic backward Euler and IS parametrizing it

Table Algorithm 1.  Inference-based schemes adaptive to large time-stepping (ISALT): detailed algorithm

 Input: Full model; a high fidelity solver preserving the invariant measure. Output: Estimated parametric scheme 1: Generate data: solve the system with the high fidelity solver, which has a small time-step $dt$; down sample to get time series with ${\delta}= \mathrm{Gap}\times dt$. Denote the data, consisting of $M$ independent trajectories on $[0,N{\delta}]$, by $\{{{\bf X}}_{t_0:t_N}^{(m)}, {{\bf B}}_{t_0:t_N}^{(m)}\}_{m=1}^M$ with $t_i= i{\delta}$. 2: Pick a parametric form approximating the flow map (2.1) as in (2.5)–(2.6). 3: Estimate parameters $c_{0:p}^{\delta}$ and $\sigma_\eta$ as in (2.7). 4: Model selection: run the inferred scheme for cross-validation, and test the consistency of the estimators.

Table 2.  Time gap of blow-up for each scheme: plain verse inferred

 1D double-well 2D gradient system 3D Lorenz system Plain RK4 ${\mathrm{Gap}}=20$ ${\mathrm{Gap}}=20$ ${\mathrm{Gap}}=10$ IS-RK4 ${\mathrm{Gap}}>200$ ${\mathrm{Gap}}>200$ ${\mathrm{Gap}}>400$ Plain SSBE ${\mathrm{Gap}}=40$ ${\mathrm{Gap}}=40$ ${\mathrm{Gap}}=20$ IS-SSBE ${\mathrm{Gap}}>200$ ${\mathrm{Gap}}>200$ ${\mathrm{Gap}}>400$

Figures(9)

Tables(3)