
-
Previous Article
Parameterization method for unstable manifolds of delay differential equations
- JCD Home
- This Issue
- Next Article
Kernel methods for the approximation of some key quantities of nonlinear systems
1. | Laboratory for Computational and Statistical Learning, Massachusetts Institute of Technology, Cambridge, MA, USA |
2. | Department of Mathematics, AlFaisal University, Riyadh, KSA |
We introduce a data-based approach to estimating key quantities which arise in the study of nonlinear control systems and random nonlinear dynamical systems. Our approach hinges on the observation that much of the existing linear theory may be readily extended to nonlinear systems -with a reasonable expectation of success -once the nonlinear system has been mapped into a high or infinite dimensional feature space. In particular, we embed a nonlinear system in a reproducing kernel Hilbert space where linear theory can be used to develop computable, non-parametric estimators approximating controllability and observability energy functions for nonlinear systems. In all cases the relevant quantities are estimated from simulated or observed data. It is then shown that the controllability energy estimator provides a key means for approximating the invariant measure of an ergodic, stochastically forced nonlinear system.
References:
[1] |
N. Aronszajn,
Theory of reproducing kernels, Trans. Amer. Math. Soc., 68 (1950), 337-404.
doi: 10.1090/S0002-9947-1950-0051437-7. |
[2] |
G. Biau, B. Cadre and B. Pelletier,
Exact rates in density support estimation, J. Multivariate Anal., 99 (2008), 2185-2207.
doi: 10.1016/j.jmva.2008.02.021. |
[3] |
V. I. Bogachev, Gaussian Measures, American Mathematical Society, 1998. |
[4] |
J. Bouvrie and B. Hamzi, Balanced reduction of nonlinear control systems in reproducing kernel Hilbert space, in Proc. 48th Annual Allerton Conference on Communication, Control, and Computing, (2010), 294-301, http://arxiv.org/abs/1011.2952.
doi: 10.1109/ALLERTON.2010.5706920. |
[5] |
J. Bouvrie and B. Hamzi, Kernel methods for the approximation of nonlinear systems, in SIAM Journal on Control and Optimization, (2017), to appear, https://arxiv.org/abs/1108.2903. Google Scholar |
[6] |
J. Bouvrie and B. Hamzi, Empirical estimators for stochastically forced nonlinear systems: Observability, controllability and the invariant measure, in Proc. American Control Conference (ACC), 2012, (2012).
doi: 10.1109/ACC.2012.6315175. |
[7] |
R. Brockett, Stochastic Control, Lecture Notes, Harvard University Press, 2009. Google Scholar |
[8] |
R. L. Butchart,
An explicit solution to the Fokker-Planck equation for an ordinary differential equation, Int. J. Control, 1 (1965), 201-208.
doi: 10.1080/00207176508905472. |
[9] |
A. Caponnetto and E. De Vito,
Optimal rates for the regularized least-squares algorithm, Found. Comput. Math., 7 (2007), 331-368.
doi: 10.1007/s10208-006-0196-8. |
[10] |
F. Cucker and S. Smale,
On the mathematical foundations of learning, Bull. AMS, 39 (2002), 1-49.
doi: 10.1090/S0273-0979-01-00923-5. |
[11] |
G. Da Prato, An Introduction to Infinite Dimensional Analysis, Springer, 2006.
doi: 10.1007/3-540-29021-4. |
[12] |
G. Da Prato and J. Zabczyk, Stochastic Equations in Infinite Dimensions, Cambridge University Press, 1992.
doi: 10.1017/CBO9780511666223. |
[13] |
E. De Vito, L. Rosasco and A. Toigo, Spectral Regularization for Support Estimation, in J. Shawe-Taylor et al., eds., Advances in Neural Information Processing Systems (NIPS), 24, Vancouver, Curran Associates, Inc., 2010. Google Scholar |
[14] |
G. E. Dullerud and F. Paganini, A Course in Robust Control Theory: A Convex Approach, Springer, 2000.
doi: 10.1007/978-1-4757-3290-0. |
[15] |
G. Froyland, K. Judd, A. I. Mees, K. Murao and D. Watson, Constructing invariant measures from data, Int. J. Bifurcat. Chaos, 5 (1995), 1181-1192. Google Scholar |
[16] |
G. Froyland, Extracting dynamical behaviour via Markov models, In Alistair Mees, ed., Nonlinear Dynamics and Statistics: Proceedings, Newton Institute, Cambridge, (2001), 281–321, Birkhauser. |
[17] |
K. Fujimoto and D. Tsubakino,
Computation of nonlinear balanced realization and model reduction based on Taylor series expansion, Systems and Control Letters, 57 (2008), 283-289.
doi: 10.1016/j.sysconle.2007.08.015. |
[18] |
A. T. Fuller,
Analysis of nonlinear stochastic systems by means of the Fokker-Planck equation, Int. J. Control, 9 (1969), 603-655.
doi: 10.1007/3-540-29021-4. |
[19] |
P. Giesl and H. Wendland,
Meshless collocation: Error estimates with application to dynamical systems, SIAM J. Num. Anal., 45 (2007), 1723-1749.
doi: 10.1137/060658813. |
[20] |
P. Giesl, Construction of Global Lyapunov Functions Using Radial Basis Functions, Springer, 2007. |
[21] |
W. S. Gray and E. I. Verriest, Algebraically defined gramians for nonlinear systems, Proc. of the 45th IEEE CDC , (2006).
doi: 10.1109/CDC.2006.376840. |
[22] |
J. Guinez, R. Quintero and A. D. Rueda,
Calculating steady states for a Fokker-Planck equation, Acta Math. Hungar., 91 (2001), 311-323.
doi: 10.1023/A:1010615818034. |
[23] |
C. Hartmann and C. Schuette,
Balancing of partially-observed stochastic differential equations, Proc. of the 47th IEEE CDC, (2008), 4867-4872.
doi: 10.1007/3-540-29021-4. |
[24] |
D. Kilminster, D. Allingham and A. Mees,
Estimating invariant probability densities for dynamical systems: Nonparametric approach to time series analysis, Ann. Ⅰ. Stat. Math., 39 (2002), 1-49.
doi: 10.1023/A:1016134209348. |
[25] |
A. J. Krener, The Important State Coordinates of a Nonlinear System, In Advances in control theory and applications, C. Bonivento, A. Isidori, L. Marconi, C. Rossi, editors, 353 (2007), 161-170, Springer.
doi: 10.1007/978-3-540-70701-1_8. |
[26] |
A. J. Krener, Reduced order modeling of nonlinear control systems, In Analysis and Design of Nonlinear Control Systems, A. Astolfi and L. Marconi, editors, (2008), 41-62, Springer.
doi: 10.1007/978-3-540-74358-3_4. |
[27] |
S. Lall, J. Marsden and S. Glavaski,
A subspace approach to balanced truncation for model reduction of nonlinear control systems, nt. J. on Robust and Nonl. Contr., 12 (2002), 519-535.
doi: 10.1002/rnc.657. |
[28] |
D. Liberzon and R. W. Brockett,
Nonlinear feedback systems perturbed by noise: Steady-state probability distributions and optimal control, IEEE T. Automat. Control, 45 (2000), 1116-1130.
doi: 10.1109/9.863596. |
[29] |
B. Moore,
Principal component analysis in linear systems: Controllability, observability, and model reduction, IEEE T. Automat. Control, 26 (1981), 17-32.
doi: 10.1109/TAC.1981.1102568. |
[30] |
A. J. Newman and P. S. Krishnaprasad, Computing balanced realizations for nonlinear systems, Proc. of the Math. Theory of Networks and Systems (MTNS), (2000). Google Scholar |
[31] |
H. Risken, The Fokker-Planck Equation, Springer, 1984.
doi: 10.1007/978-3-642-96807-5. |
[32] |
L. Rosasco, M. Belkin and E. De BVito,
On learning with integral operators, J. Mach. Learn. Res., 11 (2010), 905-934.
|
[33] |
C. W. Rowley,
Model reduction for fluids using balanced proper orthogonal decomposition, Int. J. Bifurcat. Chaos, 11 (2010), 905-934.
doi: 10.1142/S0218127405012429. |
[34] |
J. M. A Scherpen,
Balancing for nonlinear systems, Systems & Control Letters, 21 (1993), 143-153.
doi: 10.1016/0167-6911(93)90117-O. |
[35] |
B. Schölkopf and A. J. Smola, Learning with Kernels: Support Vector Machines,Regularization, Optimization, and Beyond, MIT Press, 2001. Google Scholar |
[36] |
S. Smale and D. X. Zhou,
Learning theory estimates via integral operators and their approximations, Constr. Approx., 26 (2007), 153-172.
doi: 10.1007/s00365-006-0659-y. |
[37] |
G. Wahba, Spline Models for Observational Data, SIAM CBMS-NSF Regional Conference Series in Applied Mathematics, Philadelphia, PA, 1990.
doi: 10.1137/1.9781611970128. |
[38] |
H. Wendland, Scattered Data Approximation, Cambridge Monogr. Appl. Comput. Math., Cambridge University Press, Cambridge, UK, 2005. |
[39] |
M. Zakai,
A Lyapunov criterion for the existence of stationary probability distributions for systems perturbed by noise, SIAM J. Control, 1 (1969), 390-397.
doi: 10.1137/0307028. |
show all references
References:
[1] |
N. Aronszajn,
Theory of reproducing kernels, Trans. Amer. Math. Soc., 68 (1950), 337-404.
doi: 10.1090/S0002-9947-1950-0051437-7. |
[2] |
G. Biau, B. Cadre and B. Pelletier,
Exact rates in density support estimation, J. Multivariate Anal., 99 (2008), 2185-2207.
doi: 10.1016/j.jmva.2008.02.021. |
[3] |
V. I. Bogachev, Gaussian Measures, American Mathematical Society, 1998. |
[4] |
J. Bouvrie and B. Hamzi, Balanced reduction of nonlinear control systems in reproducing kernel Hilbert space, in Proc. 48th Annual Allerton Conference on Communication, Control, and Computing, (2010), 294-301, http://arxiv.org/abs/1011.2952.
doi: 10.1109/ALLERTON.2010.5706920. |
[5] |
J. Bouvrie and B. Hamzi, Kernel methods for the approximation of nonlinear systems, in SIAM Journal on Control and Optimization, (2017), to appear, https://arxiv.org/abs/1108.2903. Google Scholar |
[6] |
J. Bouvrie and B. Hamzi, Empirical estimators for stochastically forced nonlinear systems: Observability, controllability and the invariant measure, in Proc. American Control Conference (ACC), 2012, (2012).
doi: 10.1109/ACC.2012.6315175. |
[7] |
R. Brockett, Stochastic Control, Lecture Notes, Harvard University Press, 2009. Google Scholar |
[8] |
R. L. Butchart,
An explicit solution to the Fokker-Planck equation for an ordinary differential equation, Int. J. Control, 1 (1965), 201-208.
doi: 10.1080/00207176508905472. |
[9] |
A. Caponnetto and E. De Vito,
Optimal rates for the regularized least-squares algorithm, Found. Comput. Math., 7 (2007), 331-368.
doi: 10.1007/s10208-006-0196-8. |
[10] |
F. Cucker and S. Smale,
On the mathematical foundations of learning, Bull. AMS, 39 (2002), 1-49.
doi: 10.1090/S0273-0979-01-00923-5. |
[11] |
G. Da Prato, An Introduction to Infinite Dimensional Analysis, Springer, 2006.
doi: 10.1007/3-540-29021-4. |
[12] |
G. Da Prato and J. Zabczyk, Stochastic Equations in Infinite Dimensions, Cambridge University Press, 1992.
doi: 10.1017/CBO9780511666223. |
[13] |
E. De Vito, L. Rosasco and A. Toigo, Spectral Regularization for Support Estimation, in J. Shawe-Taylor et al., eds., Advances in Neural Information Processing Systems (NIPS), 24, Vancouver, Curran Associates, Inc., 2010. Google Scholar |
[14] |
G. E. Dullerud and F. Paganini, A Course in Robust Control Theory: A Convex Approach, Springer, 2000.
doi: 10.1007/978-1-4757-3290-0. |
[15] |
G. Froyland, K. Judd, A. I. Mees, K. Murao and D. Watson, Constructing invariant measures from data, Int. J. Bifurcat. Chaos, 5 (1995), 1181-1192. Google Scholar |
[16] |
G. Froyland, Extracting dynamical behaviour via Markov models, In Alistair Mees, ed., Nonlinear Dynamics and Statistics: Proceedings, Newton Institute, Cambridge, (2001), 281–321, Birkhauser. |
[17] |
K. Fujimoto and D. Tsubakino,
Computation of nonlinear balanced realization and model reduction based on Taylor series expansion, Systems and Control Letters, 57 (2008), 283-289.
doi: 10.1016/j.sysconle.2007.08.015. |
[18] |
A. T. Fuller,
Analysis of nonlinear stochastic systems by means of the Fokker-Planck equation, Int. J. Control, 9 (1969), 603-655.
doi: 10.1007/3-540-29021-4. |
[19] |
P. Giesl and H. Wendland,
Meshless collocation: Error estimates with application to dynamical systems, SIAM J. Num. Anal., 45 (2007), 1723-1749.
doi: 10.1137/060658813. |
[20] |
P. Giesl, Construction of Global Lyapunov Functions Using Radial Basis Functions, Springer, 2007. |
[21] |
W. S. Gray and E. I. Verriest, Algebraically defined gramians for nonlinear systems, Proc. of the 45th IEEE CDC , (2006).
doi: 10.1109/CDC.2006.376840. |
[22] |
J. Guinez, R. Quintero and A. D. Rueda,
Calculating steady states for a Fokker-Planck equation, Acta Math. Hungar., 91 (2001), 311-323.
doi: 10.1023/A:1010615818034. |
[23] |
C. Hartmann and C. Schuette,
Balancing of partially-observed stochastic differential equations, Proc. of the 47th IEEE CDC, (2008), 4867-4872.
doi: 10.1007/3-540-29021-4. |
[24] |
D. Kilminster, D. Allingham and A. Mees,
Estimating invariant probability densities for dynamical systems: Nonparametric approach to time series analysis, Ann. Ⅰ. Stat. Math., 39 (2002), 1-49.
doi: 10.1023/A:1016134209348. |
[25] |
A. J. Krener, The Important State Coordinates of a Nonlinear System, In Advances in control theory and applications, C. Bonivento, A. Isidori, L. Marconi, C. Rossi, editors, 353 (2007), 161-170, Springer.
doi: 10.1007/978-3-540-70701-1_8. |
[26] |
A. J. Krener, Reduced order modeling of nonlinear control systems, In Analysis and Design of Nonlinear Control Systems, A. Astolfi and L. Marconi, editors, (2008), 41-62, Springer.
doi: 10.1007/978-3-540-74358-3_4. |
[27] |
S. Lall, J. Marsden and S. Glavaski,
A subspace approach to balanced truncation for model reduction of nonlinear control systems, nt. J. on Robust and Nonl. Contr., 12 (2002), 519-535.
doi: 10.1002/rnc.657. |
[28] |
D. Liberzon and R. W. Brockett,
Nonlinear feedback systems perturbed by noise: Steady-state probability distributions and optimal control, IEEE T. Automat. Control, 45 (2000), 1116-1130.
doi: 10.1109/9.863596. |
[29] |
B. Moore,
Principal component analysis in linear systems: Controllability, observability, and model reduction, IEEE T. Automat. Control, 26 (1981), 17-32.
doi: 10.1109/TAC.1981.1102568. |
[30] |
A. J. Newman and P. S. Krishnaprasad, Computing balanced realizations for nonlinear systems, Proc. of the Math. Theory of Networks and Systems (MTNS), (2000). Google Scholar |
[31] |
H. Risken, The Fokker-Planck Equation, Springer, 1984.
doi: 10.1007/978-3-642-96807-5. |
[32] |
L. Rosasco, M. Belkin and E. De BVito,
On learning with integral operators, J. Mach. Learn. Res., 11 (2010), 905-934.
|
[33] |
C. W. Rowley,
Model reduction for fluids using balanced proper orthogonal decomposition, Int. J. Bifurcat. Chaos, 11 (2010), 905-934.
doi: 10.1142/S0218127405012429. |
[34] |
J. M. A Scherpen,
Balancing for nonlinear systems, Systems & Control Letters, 21 (1993), 143-153.
doi: 10.1016/0167-6911(93)90117-O. |
[35] |
B. Schölkopf and A. J. Smola, Learning with Kernels: Support Vector Machines,Regularization, Optimization, and Beyond, MIT Press, 2001. Google Scholar |
[36] |
S. Smale and D. X. Zhou,
Learning theory estimates via integral operators and their approximations, Constr. Approx., 26 (2007), 153-172.
doi: 10.1007/s00365-006-0659-y. |
[37] |
G. Wahba, Spline Models for Observational Data, SIAM CBMS-NSF Regional Conference Series in Applied Mathematics, Philadelphia, PA, 1990.
doi: 10.1137/1.9781611970128. |
[38] |
H. Wendland, Scattered Data Approximation, Cambridge Monogr. Appl. Comput. Math., Cambridge University Press, Cambridge, UK, 2005. |
[39] |
M. Zakai,
A Lyapunov criterion for the existence of stationary probability distributions for systems perturbed by noise, SIAM J. Control, 1 (1969), 390-397.
doi: 10.1137/0307028. |

[1] |
Gervy Marie Angeles, Gilbert Peralta. Energy method for exponential stability of coupled one-dimensional hyperbolic PDE-ODE systems. Evolution Equations & Control Theory, 2020 doi: 10.3934/eect.2020108 |
[2] |
Xuefei He, Kun Wang, Liwei Xu. Efficient finite difference methods for the nonlinear Helmholtz equation in Kerr medium. Electronic Research Archive, 2020, 28 (4) : 1503-1528. doi: 10.3934/era.2020079 |
[3] |
Xiaoxiao Li, Yingjing Shi, Rui Li, Shida Cao. Energy management method for an unpowered landing. Journal of Industrial & Management Optimization, 2020 doi: 10.3934/jimo.2020180 |
[4] |
Manuel del Pino, Monica Musso, Juncheng Wei, Yifu Zhou. Type Ⅱ finite time blow-up for the energy critical heat equation in $ \mathbb{R}^4 $. Discrete & Continuous Dynamical Systems - A, 2020, 40 (6) : 3327-3355. doi: 10.3934/dcds.2020052 |
[5] |
Kengo Nakai, Yoshitaka Saiki. Machine-learning construction of a model for a macroscopic fluid variable using the delay-coordinate of a scalar observable. Discrete & Continuous Dynamical Systems - S, 2021, 14 (3) : 1079-1092. doi: 10.3934/dcdss.2020352 |
[6] |
Xinfu Chen, Huiqiang Jiang, Guoqing Liu. Boundary spike of the singular limit of an energy minimizing problem. Discrete & Continuous Dynamical Systems - A, 2020, 40 (6) : 3253-3290. doi: 10.3934/dcds.2020124 |
[7] |
Marcello D'Abbicco, Giovanni Girardi, Giséle Ruiz Goldstein, Jerome A. Goldstein, Silvia Romanelli. Equipartition of energy for nonautonomous damped wave equations. Discrete & Continuous Dynamical Systems - S, 2021, 14 (2) : 597-613. doi: 10.3934/dcdss.2020364 |
[8] |
Luis Caffarelli, Fanghua Lin. Nonlocal heat flows preserving the L2 energy. Discrete & Continuous Dynamical Systems - A, 2009, 23 (1&2) : 49-64. doi: 10.3934/dcds.2009.23.49 |
[9] |
Awais Younus, Zoubia Dastgeer, Nudrat Ishaq, Abdul Ghaffar, Kottakkaran Sooppy Nisar, Devendra Kumar. On the observability of conformable linear time-invariant control systems. Discrete & Continuous Dynamical Systems - S, 2020 doi: 10.3934/dcdss.2020444 |
[10] |
Oleg Yu. Imanuvilov, Jean Pierre Puel. On global controllability of 2-D Burgers equation. Discrete & Continuous Dynamical Systems - A, 2009, 23 (1&2) : 299-313. doi: 10.3934/dcds.2009.23.299 |
[11] |
Julian Tugaut. Captivity of the solution to the granular media equation. Kinetic & Related Models, , () : -. doi: 10.3934/krm.2021002 |
[12] |
Tong Yang, Seiji Ukai, Huijiang Zhao. Stationary solutions to the exterior problems for the Boltzmann equation, I. Existence. Discrete & Continuous Dynamical Systems - A, 2009, 23 (1&2) : 495-520. doi: 10.3934/dcds.2009.23.495 |
[13] |
Xiaoming Wang. Upper semi-continuity of stationary statistical properties of dissipative systems. Discrete & Continuous Dynamical Systems - A, 2009, 23 (1&2) : 521-540. doi: 10.3934/dcds.2009.23.521 |
[14] |
Predrag S. Stanimirović, Branislav Ivanov, Haifeng Ma, Dijana Mosić. A survey of gradient methods for solving nonlinear optimization. Electronic Research Archive, 2020, 28 (4) : 1573-1624. doi: 10.3934/era.2020115 |
[15] |
Oussama Landoulsi. Construction of a solitary wave solution of the nonlinear focusing schrödinger equation outside a strictly convex obstacle in the $ L^2 $-supercritical case. Discrete & Continuous Dynamical Systems - A, 2021, 41 (2) : 701-746. doi: 10.3934/dcds.2020298 |
[16] |
Bo Chen, Youde Wang. Global weak solutions for Landau-Lifshitz flows and heat flows associated to micromagnetic energy functional. Communications on Pure & Applied Analysis, 2021, 20 (1) : 319-338. doi: 10.3934/cpaa.2020268 |
[17] |
Kuo-Chih Hung, Shin-Hwa Wang. Classification and evolution of bifurcation curves for a porous-medium combustion problem with large activation energy. Communications on Pure & Applied Analysis, , () : -. doi: 10.3934/cpaa.2020281 |
[18] |
Tianwen Luo, Tao Tao, Liqun Zhang. Finite energy weak solutions of 2d Boussinesq equations with diffusive temperature. Discrete & Continuous Dynamical Systems - A, 2020, 40 (6) : 3737-3765. doi: 10.3934/dcds.2019230 |
[19] |
Kai Yang. Scattering of the focusing energy-critical NLS with inverse square potential in the radial case. Communications on Pure & Applied Analysis, 2021, 20 (1) : 77-99. doi: 10.3934/cpaa.2020258 |
[20] |
Md. Masum Murshed, Kouta Futai, Masato Kimura, Hirofumi Notsu. Theoretical and numerical studies for energy estimates of the shallow water equations with a transmission boundary condition. Discrete & Continuous Dynamical Systems - S, 2021, 14 (3) : 1063-1078. doi: 10.3934/dcdss.2020230 |
Impact Factor:
Tools
Metrics
Other articles
by authors
[Back to Top]