Data-driven evolutions of critical points

Stefano Almi; Massimo Fornasier; Richard Huber

doi:10.3934/fods.2020011

Previous Article
Spectral methods to study the robustness of residual neural networks with infinite layers
FoDS Home
This Issue
Next Article

2020, 2(3): 207-255. doi: 10.3934/fods.2020011

Data-driven evolutions of critical points

Stefano Almi ^1,, , Massimo Fornasier ^2, and Richard Huber ^3,

1.	Department of Mathematics, University of Vienna, Oskar-Morgenstern-Platz 1, 1090 Vienna, Austria
2.	Department of Mathematics, Technical University of Munich, Boltzmannstrasse 3, 85748 Garching bei München, Germany
3.	Institute of Mathematics and Scientific Computing, Karl-Franzens University of Graz, Heinrichstrasse 36/III, 8010 Graz, Austria

^* Corresponding author

Published 2020 Early access August 2020

Fund Project: S.A. acknowledges the support of the DFG Transregio 109 "Discretization in Geometry and Dynamics". M.F. acknowledges the support of the DFG Project "Identification of Energies from Observation of Evolutions" and the DFG SPP 1962 "Non-smooth and Complementarity-based Distributed Parameter Systems: Simulation and Hierarchical Optimization". R.H. acknowledges the support of the DFG-FWF IGDK1754 "Optimization and Numerical Analysis for Partial Differential Equations with Nonsmooth Structures" and the hospitality of TUM during the preparation of this work

In this paper we are concerned with the learnability of energies from data obtained by observing time evolutions of their critical points starting at random initial equilibria. As a byproduct of our theoretical framework we introduce the novel concept of mean-field limit of critical point evolutions and of their energy balance as a new form of transport. We formulate the energy learning as a variational problem, minimizing the discrepancy of energy competitors from fulfilling the equilibrium condition along any trajectory of critical points originated at random initial equilibria. By $ \Gamma $-convergence arguments we prove the convergence of minimal solutions obtained from finite number of observations to the exact energy in a suitable sense. The abstract framework is actually fully constructive and numerically implementable. Hence, the approximation of the energy from a finite number of observations of past evolutions allows one to simulate further evolutions, which are fully data-driven. As we aim at a precise quantitative analysis, and to provide concrete examples of tractable solutions, we present analytic and numerical results on the reconstruction of an elastic energy for a one-dimensional model of thin nonlinear-elastic rod.

Keywords: Energy learning, data assimilation, quasi-static evolutions of critical points, mean-field limit, variational calculus, probability measure transport.

Mathematics Subject Classification: Primary: 49N80, 68T05; Secondary: 34A55, 70F17.

Citation: Stefano Almi, Massimo Fornasier, Richard Huber. Data-driven evolutions of critical points. Foundations of Data Science, 2020, 2 (3) : 207-255. doi: 10.3934/fods.2020011

References:

[1]	V. Agostiniani and R. Rossi, Singular vanishing-viscosity limits of gradient flows: The finite-dimensional case, J. Differential Equations, 263 (2017), 7815-7855. doi: 10.1016/j.jde.2017.08.027.
[2]	V. Agostiniani, R. Rossi and G. Savaré, On the transversality conditions and their genericity, Rend. Circ. Mat. Palermo, 64 (2015), 101-116. doi: 10.1007/s12215-014-0184-4.
[3]	V. Agostiniani, R. Rossi, and G. Savaré, Singular vanishing-viscosity limits of gradient flows in Hilbert spaces, personal communication: in preparation, (2018).
[4]	V. Albani, U. M. Ascher, X. Yang and J. P. Zubelli, Data driven recovery of local volatility surfaces, Inverse Probl. Imaging, 11 (2017), 799-823. doi: 10.3934/ipi.2017038.
[5]	L. Ambrosio, N. Fusco and D. Pallara, Functions of Bounded Variation and Free Discontinuity Problems, Oxford Mathematical Monographs. The Clarendon Press, Oxford University Press, New York, 2000.
[6]	L. Ambrosio, N. Gigli and G. Savaré, Gradient flows in metric spaces and in the space of probability measures, Lectures in Mathematics ETH Zürich, Birkhäuser Verlag, Basel, second ed., 2008.
[7]	M. Bongini, M. Fornasier, M. Hansen and M. Maggioni, Inferring interaction rules from observations of evolutive systems Ⅰ: The variational approach, Math. Models Methods Appl. Sci., 27 (2017), 909-951. doi: 10.1142/S0218202517500208.
[8]	A. Braides, Γ-Convergence for Beginners, Oxford Lecture Series in Mathematics and its Applications, 22. Oxford University Press, Oxford, 2002. doi: 10.1093/acprof:oso/9780198507840.001.0001.
[9]	T. Q. Chen, Y. Rubanova, J. Bettencourt and D. K. Duvenaud, Neural ordinary differential equations, Advances in Neural Information Processing Systems, S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi and R. Garnett, eds., Curran Associates, Inc., 31 (2018), 6571–6583.
[10]	S. Conti, S. Müller and M. Ortiz, Data-driven problems in elasticity, Arch. Ration. Mech. Anal., 229 (2018), 79-123. doi: 10.1007/s00205-017-1214-0.
[11]	S. Crépey, Calibration of the local volatility in a generalized Black-Scholes model using Tikhonov regularization, SIAM J. Math. Anal., 34 (2003), 1183-1206. doi: 10.1137/S0036141001400202.
[12]	T. S. Cubitt, J. Eisert and M. M. Wolf, Extracting dynamical equations from experimental data is NP hard, Phys. Rev. Lett., 108 (2012), 120503. doi: 10.1103/PhysRevLett.108.120503.
[13]	G. Dal Maso, An Introduction to $\Gamma$-Convergence, vol. 8 of Progress in Nonlinear Differential Equations and their Applications, Birkhäuser Boston, Inc., Boston, MA, 1993. doi: 10.1007/978-1-4612-0327-8.
[14]	W. E, A proposal on machine learning via dynamical systems, Commun. Math. Stat., 5 (2017), 1-11. doi: 10.1007/s40304-017-0103-z.
[15]	W. E, J. Han and Q. Li, A mean-field optimal control formulation of deep learning, Res. Math. Sci., 6 (2019), No. 10, 41 pp. doi: 10.1007/s40687-018-0172-y.
[16]	H. Egger and H. W. Engl, Tikhonov regularization applied to the inverse problem of option pricing: Convergence analysis and rates, Inverse Problems, 21 (2005), 1027-1045. doi: 10.1088/0266-5611/21/3/014.
[17]	R. Gerstberger and P. Rentrop, Feedforward neural nets as discretization schemes for ODES and DAES, 7th ICCAM 96 Congress., J. Comput. Appl. Math., 82 (1997), 117-128. doi: 10.1016/S0377-0427(97)00085-X.
[18]	M. C. Grant and S. P. Boyd, Graph implementations for nonsmooth convex programs, Recent Advances in Learning and Control, V. Blondel, S. Boyd, and H. Kimura, eds., Lecture Notes in Control and Information Sciences, Springer-Verlag Limited, 371 (2008), 95–110. doi: 10.1007/978-1-84800-155-8_7.
[19]	M. Grant and S. Boyd, CVX: Matlab Software for Disciplined Convex Programming, version 2.1., http://cvxr.com/cvx, Mar. 2014.
[20]	J. Guckenheimer and P. Holmes, Nonlinear Oscillations, Dynamical Systems, and Bifurcations of Vector Fields, vol. 42 of Applied Mathematical Sciences, Springer-Verlag, New York, 1983. doi: 10.1007/978-1-4612-1140-2.
[21]	T. Kirchdoerfer and M. Ortiz, Data-driven computational mechanics, Comput. Methods Appl. Mech. Engrg., 304 (2016), 81-101. doi: 10.1016/j.cma.2016.02.001.
[22]	F. Lu, M. Maggioni and S. Tang, Learning interaction kernels in heterogeneous systems of agents from multiple trajectories, arXiv: 1910.04832.
[23]	F. Lu, M. Zhong, S. Tang and M. Maggioni, Nonparametric inference of interaction laws in systems of agents from trajectory data, Proc. Natl. Acad. Sci. USA, 116 (2019), 14424-14433. doi: 10.1073/pnas.1822012116.
[24]	E. Novak and H. Woźniakowski, Approximation of infinitely differentiable multivariate functions is intractable, J. Complexity, 25 (2009), 398-404. doi: 10.1016/j.jco.2008.11.002.
[25]	H. Schaeffer, G. Tran and R. Ward, Extracting sparse high-dimensional dynamics from limited data, SIAM J. Appl. Math., 78 (2018), 3279-3295. doi: 10.1137/18M116798X.
[26]	H. Schaeffer, G. Tran and R. Ward, Learning dynamical systems and bifurcation via group sparsity, arXiv: 1709.01558.
[27]	G. Scilla and F. Solombrino, Delayed loss of stability in singularly perturbed finite-dimensional gradient flows, Asymptot. Anal., 110 (2018), 1-19. doi: 10.3233/ASY-181475.
[28]	G. Scilla and F. Solombrino, Multiscale analysis of singularly perturbed finite dimensional gradient flows: The minimizing movement approach, Nonlinearity, 31 (2018), 5036-5074. doi: 10.1088/1361-6544/aad6ac.
[29]	G. Tran and R. Ward, Exact recovery of chaotic systems from highly corrupted data, Multiscale Model. Simul., 15 (2017), 1108-1129. doi: 10.1137/16M1086637.
[30]	C. Zanini, Singular perturbations of finite dimensional gradient flows, Discrete Contin. Dyn. Syst., 18 (2007), 657-675. doi: 10.3934/dcds.2007.18.657.
[31]	M. Zhong, J. Miller and M. Maggioni, Data-driven discovery of emergent behaviors in collective dynamics, Phys. D, 411 (2020), 132542, 25 pp. doi: 10.1016/j.physd.2020.132542.

show all references

References:

[1]	V. Agostiniani and R. Rossi, Singular vanishing-viscosity limits of gradient flows: The finite-dimensional case, J. Differential Equations, 263 (2017), 7815-7855. doi: 10.1016/j.jde.2017.08.027.
[2]	V. Agostiniani, R. Rossi and G. Savaré, On the transversality conditions and their genericity, Rend. Circ. Mat. Palermo, 64 (2015), 101-116. doi: 10.1007/s12215-014-0184-4.
[3]	V. Agostiniani, R. Rossi, and G. Savaré, Singular vanishing-viscosity limits of gradient flows in Hilbert spaces, personal communication: in preparation, (2018).
[4]	V. Albani, U. M. Ascher, X. Yang and J. P. Zubelli, Data driven recovery of local volatility surfaces, Inverse Probl. Imaging, 11 (2017), 799-823. doi: 10.3934/ipi.2017038.
[5]	L. Ambrosio, N. Fusco and D. Pallara, Functions of Bounded Variation and Free Discontinuity Problems, Oxford Mathematical Monographs. The Clarendon Press, Oxford University Press, New York, 2000.
[6]	L. Ambrosio, N. Gigli and G. Savaré, Gradient flows in metric spaces and in the space of probability measures, Lectures in Mathematics ETH Zürich, Birkhäuser Verlag, Basel, second ed., 2008.
[7]	M. Bongini, M. Fornasier, M. Hansen and M. Maggioni, Inferring interaction rules from observations of evolutive systems Ⅰ: The variational approach, Math. Models Methods Appl. Sci., 27 (2017), 909-951. doi: 10.1142/S0218202517500208.
[8]	A. Braides, Γ-Convergence for Beginners, Oxford Lecture Series in Mathematics and its Applications, 22. Oxford University Press, Oxford, 2002. doi: 10.1093/acprof:oso/9780198507840.001.0001.
[9]	T. Q. Chen, Y. Rubanova, J. Bettencourt and D. K. Duvenaud, Neural ordinary differential equations, Advances in Neural Information Processing Systems, S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi and R. Garnett, eds., Curran Associates, Inc., 31 (2018), 6571–6583.
[10]	S. Conti, S. Müller and M. Ortiz, Data-driven problems in elasticity, Arch. Ration. Mech. Anal., 229 (2018), 79-123. doi: 10.1007/s00205-017-1214-0.
[11]	S. Crépey, Calibration of the local volatility in a generalized Black-Scholes model using Tikhonov regularization, SIAM J. Math. Anal., 34 (2003), 1183-1206. doi: 10.1137/S0036141001400202.
[12]	T. S. Cubitt, J. Eisert and M. M. Wolf, Extracting dynamical equations from experimental data is NP hard, Phys. Rev. Lett., 108 (2012), 120503. doi: 10.1103/PhysRevLett.108.120503.
[13]	G. Dal Maso, An Introduction to $\Gamma$-Convergence, vol. 8 of Progress in Nonlinear Differential Equations and their Applications, Birkhäuser Boston, Inc., Boston, MA, 1993. doi: 10.1007/978-1-4612-0327-8.
[14]	W. E, A proposal on machine learning via dynamical systems, Commun. Math. Stat., 5 (2017), 1-11. doi: 10.1007/s40304-017-0103-z.
[15]	W. E, J. Han and Q. Li, A mean-field optimal control formulation of deep learning, Res. Math. Sci., 6 (2019), No. 10, 41 pp. doi: 10.1007/s40687-018-0172-y.
[16]	H. Egger and H. W. Engl, Tikhonov regularization applied to the inverse problem of option pricing: Convergence analysis and rates, Inverse Problems, 21 (2005), 1027-1045. doi: 10.1088/0266-5611/21/3/014.
[17]	R. Gerstberger and P. Rentrop, Feedforward neural nets as discretization schemes for ODES and DAES, 7th ICCAM 96 Congress., J. Comput. Appl. Math., 82 (1997), 117-128. doi: 10.1016/S0377-0427(97)00085-X.
[18]	M. C. Grant and S. P. Boyd, Graph implementations for nonsmooth convex programs, Recent Advances in Learning and Control, V. Blondel, S. Boyd, and H. Kimura, eds., Lecture Notes in Control and Information Sciences, Springer-Verlag Limited, 371 (2008), 95–110. doi: 10.1007/978-1-84800-155-8_7.
[19]	M. Grant and S. Boyd, CVX: Matlab Software for Disciplined Convex Programming, version 2.1., http://cvxr.com/cvx, Mar. 2014.
[20]	J. Guckenheimer and P. Holmes, Nonlinear Oscillations, Dynamical Systems, and Bifurcations of Vector Fields, vol. 42 of Applied Mathematical Sciences, Springer-Verlag, New York, 1983. doi: 10.1007/978-1-4612-1140-2.
[21]	T. Kirchdoerfer and M. Ortiz, Data-driven computational mechanics, Comput. Methods Appl. Mech. Engrg., 304 (2016), 81-101. doi: 10.1016/j.cma.2016.02.001.
[22]	F. Lu, M. Maggioni and S. Tang, Learning interaction kernels in heterogeneous systems of agents from multiple trajectories, arXiv: 1910.04832.
[23]	F. Lu, M. Zhong, S. Tang and M. Maggioni, Nonparametric inference of interaction laws in systems of agents from trajectory data, Proc. Natl. Acad. Sci. USA, 116 (2019), 14424-14433. doi: 10.1073/pnas.1822012116.
[24]	E. Novak and H. Woźniakowski, Approximation of infinitely differentiable multivariate functions is intractable, J. Complexity, 25 (2009), 398-404. doi: 10.1016/j.jco.2008.11.002.
[25]	H. Schaeffer, G. Tran and R. Ward, Extracting sparse high-dimensional dynamics from limited data, SIAM J. Appl. Math., 78 (2018), 3279-3295. doi: 10.1137/18M116798X.
[26]	H. Schaeffer, G. Tran and R. Ward, Learning dynamical systems and bifurcation via group sparsity, arXiv: 1709.01558.
[27]	G. Scilla and F. Solombrino, Delayed loss of stability in singularly perturbed finite-dimensional gradient flows, Asymptot. Anal., 110 (2018), 1-19. doi: 10.3233/ASY-181475.
[28]	G. Scilla and F. Solombrino, Multiscale analysis of singularly perturbed finite dimensional gradient flows: The minimizing movement approach, Nonlinearity, 31 (2018), 5036-5074. doi: 10.1088/1361-6544/aad6ac.
[29]	G. Tran and R. Ward, Exact recovery of chaotic systems from highly corrupted data, Multiscale Model. Simul., 15 (2017), 1108-1129. doi: 10.1137/16M1086637.
[30]	C. Zanini, Singular perturbations of finite dimensional gradient flows, Discrete Contin. Dyn. Syst., 18 (2007), 657-675. doi: 10.3934/dcds.2007.18.657.
[31]	M. Zhong, J. Miller and M. Maggioni, Data-driven discovery of emergent behaviors in collective dynamics, Phys. D, 411 (2020), 132542, 25 pp. doi: 10.1016/j.physd.2020.132542.

Figure 1. Reconstructions $ \hat a $ (left) and $ \hat a' $ (right) in dotted red and true $ a $ and $ a' $ in blue. The underlying red circles depict the (adaptive) nodes in $ \Lambda $. Below you see the distribution $ \tilde \eta $ of available data

Figure Options

Download full-size image

Download as PowerPoint slide

Figure 2. Reconstruction of $ a' $ with increasing number $ {N_e} $ of measurements. The true $ a' $ is depicted by the blue curve while the red line depicts the reconstruction $ \hat a' $ following (5.23). The red circles at the bottom of the figures depict the position of nodes of the underlying mesh $ \Lambda_N $. The histogram below describes the density of the available information encoded by the probability measure $ \tilde{\eta} $ for the case $ N_e = 60 $

Figure Options

Download full-size image

Download as PowerPoint slide

Figure 3. Reconstruction of $ a' $ for varying $ N $ and $ D(N) = 4N $ and distribution $ \tilde \eta $ for $ N = 60 $. Graphic as described in Figure 2

Figure Options

Download full-size image

Download as PowerPoint slide

Figure 4. Reconstruction of $ a' $ for varying $ N $ and fixed $ D(N) = 300 $ and distribution $ \tilde \eta $. Graphics as described in Figure 2

Figure Options

Download full-size image

Download as PowerPoint slide

Figure 5. Impact of different constants $ M_2 $ for constraints on $ a'' $ with $ \|a''\| \leq [2,5;20,1000] $ on reconstructs are depicted, showing improved approximations for increasing $ M_2 $. Graphics as in Figure 2

Figure Options

Download full-size image

Download as PowerPoint slide

Figure 6. Impact of changing constraint $ M_1 $ bounding the values of $ a' $, showing a projection-like behaviour of the reconstruction for small $ M_1 $. Graphics as in Figure 2

Figure Options

Download full-size image

Download as PowerPoint slide

Figure 7. Left showing $ \hat x_\varepsilon(t) $ (dark dashed) and $ x_\varepsilon(t) $ (line bright) with data from Section 5.2.2 at times $ t = [0.2 $, $ 0.4 $, $ 0.6 $, $ 0.8 $, $ 1.0] $, right with data from Section 5.2.3

Figure Options

Download full-size image

Download as PowerPoint slide

[1]	Patrick Gerard, Christophe Pallard. A mean-field toy model for resonant transport. Kinetic and Related Models, 2010, 3 (2) : 299-309. doi: 10.3934/krm.2010.3.299
[2]	Alice Fiaschi. Young-measure quasi-static damage evolution: The nonconvex and the brittle cases. Discrete and Continuous Dynamical Systems - S, 2013, 6 (1) : 17-42. doi: 10.3934/dcdss.2013.6.17
[3]	Roman VodiČka, Vladislav MantiČ. An energy based formulation of a quasi-static interface damage model with a multilinear cohesive law. Discrete and Continuous Dynamical Systems - S, 2017, 10 (6) : 1539-1561. doi: 10.3934/dcdss.2017079
[4]	Seung-Yeal Ha, Jinwook Jung, Jeongho Kim, Jinyeong Park, Xiongtao Zhang. A mean-field limit of the particle swarmalator model. Kinetic and Related Models, 2021, 14 (3) : 429-468. doi: 10.3934/krm.2021011
[5]	Dorothee Knees, Andreas Schröder. Computational aspects of quasi-static crack propagation. Discrete and Continuous Dynamical Systems - S, 2013, 6 (1) : 63-99. doi: 10.3934/dcdss.2013.6.63
[6]	Przemysław Górka. Quasi-static evolution of polyhedral crystals. Discrete and Continuous Dynamical Systems - B, 2008, 9 (2) : 309-320. doi: 10.3934/dcdsb.2008.9.309
[7]	Gerasimenko Viktor. Heisenberg picture of quantum kinetic evolution in mean-field limit. Kinetic and Related Models, 2011, 4 (1) : 385-399. doi: 10.3934/krm.2011.4.385
[8]	Seung-Yeal Ha, Jeongho Kim, Jinyeong Park, Xiongtao Zhang. Uniform stability and mean-field limit for the augmented Kuramoto model. Networks and Heterogeneous Media, 2018, 13 (2) : 297-322. doi: 10.3934/nhm.2018013
[9]	Michael Herty, Mattia Zanella. Performance bounds for the mean-field limit of constrained dynamics. Discrete and Continuous Dynamical Systems, 2017, 37 (4) : 2023-2043. doi: 10.3934/dcds.2017086
[10]	Nastassia Pouradier Duteil. Mean-field limit of collective dynamics with time-varying weights. Networks and Heterogeneous Media, 2022, 17 (2) : 129-161. doi: 10.3934/nhm.2022001
[11]	Matthew Rosenzweig. The mean-field limit of the Lieb-Liniger model. Discrete and Continuous Dynamical Systems, 2022, 42 (6) : 3005-3037. doi: 10.3934/dcds.2022006
[12]	Irina F. Sivergina, Michael P. Polis. About global null controllability of a quasi-static thermoelastic contact system. Conference Publications, 2005, 2005 (Special) : 816-823. doi: 10.3934/proc.2005.2005.816
[13]	Christopher J. Larsen. Local minimality and crack prediction in quasi-static Griffith fracture evolution. Discrete and Continuous Dynamical Systems - S, 2013, 6 (1) : 121-129. doi: 10.3934/dcdss.2013.6.121
[14]	Xianping Wu, Xun Li, Zhongfei Li. A mean-field formulation for multi-period asset-liability mean-variance portfolio selection with probability constraints. Journal of Industrial and Management Optimization, 2018, 14 (1) : 249-265. doi: 10.3934/jimo.2017045
[15]	Jochen Bröcker. Existence and uniqueness for variational data assimilation in continuous time. Mathematical Control and Related Fields, 2021 doi: 10.3934/mcrf.2021050
[16]	Seung-Yeal Ha, Jeongho Kim, Peter Pickl, Xiongtao Zhang. A probabilistic approach for the mean-field limit to the Cucker-Smale model with a singular communication. Kinetic and Related Models, 2019, 12 (5) : 1045-1067. doi: 10.3934/krm.2019039
[17]	Young-Pil Choi, Samir Salem. Cucker-Smale flocking particles with multiplicative noises: Stochastic mean-field limit and phase transition. Kinetic and Related Models, 2019, 12 (3) : 573-592. doi: 10.3934/krm.2019023
[18]	Seung-Yeal Ha, Jeongho Kim, Xiongtao Zhang. Uniform stability of the Cucker-Smale model and its application to the Mean-Field limit. Kinetic and Related Models, 2018, 11 (5) : 1157-1181. doi: 10.3934/krm.2018045
[19]	Joachim Crevat. Mean-field limit of a spatially-extended FitzHugh-Nagumo neural network. Kinetic and Related Models, 2019, 12 (6) : 1329-1358. doi: 10.3934/krm.2019052
[20]	Rong Yang, Li Chen. Mean-field limit for a collision-avoiding flocking system and the time-asymptotic flocking dynamics for the kinetic equation. Kinetic and Related Models, 2014, 7 (2) : 381-400. doi: 10.3934/krm.2014.7.381

Impact Factor:

Tools

Metrics

Tools

Article outline

Figures and Tables

Impact Factor:

Tools

Figures and Tables

Impact Factor:

American Institute of Mathematical Sciences

Data-driven evolutions of critical points

References:

References:

Tools

Metrics

Other articles
by authors

Tools

Tools

Tools

Metrics

Other articles
by authors

American Institute of Mathematical Sciences

Data-driven evolutions of critical points

References:

References:

Tools

Metrics

Other articlesby authors

Tools

Tools

Tools

Metrics

Other articlesby authors

Other articles
by authors

Other articles
by authors