-
Previous Article
Learning nonlocal regularization operators
- MCRF Home
- This Issue
-
Next Article
Noise and stability in reaction-diffusion equations
Stochastic optimal control — A concise introduction
Department of Mathematics, University of Central Florida, Orlando, FL 32816, USA |
This is a concise introduction to stochastic optimal control theory. We assume that the readers have basic knowledge of real analysis, functional analysis, elementary probability, ordinary differential equations and partial differential equations. We will present the following topics: (ⅰ) A brief presentation of relevant results on stochastic analysis; (ⅱ) Formulation of stochastic optimal control problems; (ⅲ) Variational method and Pontryagin's maximum principle, together with a brief introduction of backward stochastic differential equations; (ⅳ) Dynamic programming method and viscosity solutions to Hamilton-Jacobi-Bellman equation; (ⅴ) Linear-quadratic optimal control problems, including a careful discussion on open-loop optimal controls and closed-loop optimal strategies, linear forward-backward stochastic differential equations, and Riccati equations.
References:
[1] |
M. G. Crandall and P. L. Lions,
Viscosity solutions of Hamilton-Jacobi equations, Trans. Amer. Math. Soc., 277 (1983), 1-42.
doi: 10.1090/S0002-9947-1983-0690039-8. |
[2] |
L. C. Evans and R. F. Gariepy, Measure Theory and Fine Properties of Functions, CRC Press, Boca Raton, FL, 1992.
![]() |
[3] |
W. H. Fleming and H. M. Soner, Controlled Markov Processes and Viscosity Solutions, Springer-Verlag, New York, 1993. |
[4] |
A. Gary, D. Greenhalgh, L. Hu, X. Mao and J. Pan,
A stochastic differential equation SIS epidemic model, SIAM J. Appl. Math., 71 (2011), 876-902.
doi: 10.1137/10081856X. |
[5] |
S. He, J. Wang and J. Yan, Semimartingale Theory and Stochastic Calculus, Science Press and CRC Press, Beijing, 1992.
![]() |
[6] |
I. Karatzas and S. Shreve, Brownian Motion and Stochastic Calculus, Springer-Verlag, New York, 1988, 47–127.
doi: 10.1007/978-1-4684-0302-2_2. |
[7] |
E. Pardoux and S. Peng,
Adapted solution of backward stochastic differential equations, Systems Control Lett., 14 (1990), 55-61.
doi: 10.1016/0167-6911(90)90082-6. |
[8] |
S. Peng,
A general stochastic maximum principle for optimal control problems, SIAM J. Control Optim., 28 (1990), 966-979.
doi: 10.1137/0328054. |
[9] |
J. Sun, X. Li and J. Yong,
Open-loop and closed-loop solvabilities for stochastic linear quadratic optimal control problems, SIAM J. Control Optim., 54 (2016), 2274-2308.
doi: 10.1137/15M103532X. |
[10] |
J. Sun and J. Yong,
Linear quadratic stochastic differential games: Open-loop and closed-loop saddle points, SIAM J. Control Optim., 52 (2014), 4082-4121.
doi: 10.1137/140953642. |
[11] |
E. Tornatore, S. M. Buccellato and P. Vetro,
Stability of a stochastic SIR system, Physica A, 354 (2005), 111-126.
doi: 10.1016/j.physa.2005.02.057. |
[12] |
J. Yong and X. Y. Zhou, Stochastic Controls: Hamiltonian Systems and HJB Equations, Springer-Verlag, New York, 1999.
doi: 10.1007/978-1-4612-1466-3. |
show all references
References:
[1] |
M. G. Crandall and P. L. Lions,
Viscosity solutions of Hamilton-Jacobi equations, Trans. Amer. Math. Soc., 277 (1983), 1-42.
doi: 10.1090/S0002-9947-1983-0690039-8. |
[2] |
L. C. Evans and R. F. Gariepy, Measure Theory and Fine Properties of Functions, CRC Press, Boca Raton, FL, 1992.
![]() |
[3] |
W. H. Fleming and H. M. Soner, Controlled Markov Processes and Viscosity Solutions, Springer-Verlag, New York, 1993. |
[4] |
A. Gary, D. Greenhalgh, L. Hu, X. Mao and J. Pan,
A stochastic differential equation SIS epidemic model, SIAM J. Appl. Math., 71 (2011), 876-902.
doi: 10.1137/10081856X. |
[5] |
S. He, J. Wang and J. Yan, Semimartingale Theory and Stochastic Calculus, Science Press and CRC Press, Beijing, 1992.
![]() |
[6] |
I. Karatzas and S. Shreve, Brownian Motion and Stochastic Calculus, Springer-Verlag, New York, 1988, 47–127.
doi: 10.1007/978-1-4684-0302-2_2. |
[7] |
E. Pardoux and S. Peng,
Adapted solution of backward stochastic differential equations, Systems Control Lett., 14 (1990), 55-61.
doi: 10.1016/0167-6911(90)90082-6. |
[8] |
S. Peng,
A general stochastic maximum principle for optimal control problems, SIAM J. Control Optim., 28 (1990), 966-979.
doi: 10.1137/0328054. |
[9] |
J. Sun, X. Li and J. Yong,
Open-loop and closed-loop solvabilities for stochastic linear quadratic optimal control problems, SIAM J. Control Optim., 54 (2016), 2274-2308.
doi: 10.1137/15M103532X. |
[10] |
J. Sun and J. Yong,
Linear quadratic stochastic differential games: Open-loop and closed-loop saddle points, SIAM J. Control Optim., 52 (2014), 4082-4121.
doi: 10.1137/140953642. |
[11] |
E. Tornatore, S. M. Buccellato and P. Vetro,
Stability of a stochastic SIR system, Physica A, 354 (2005), 111-126.
doi: 10.1016/j.physa.2005.02.057. |
[12] |
J. Yong and X. Y. Zhou, Stochastic Controls: Hamiltonian Systems and HJB Equations, Springer-Verlag, New York, 1999.
doi: 10.1007/978-1-4612-1466-3. |
[1] |
Diana Keller. Optimal control of a linear stochastic Schrödinger equation. Conference Publications, 2013, 2013 (special) : 437-446. doi: 10.3934/proc.2013.2013.437 |
[2] |
Shanjian Tang, Fu Zhang. Path-dependent optimal stochastic control and viscosity solution of associated Bellman equations. Discrete & Continuous Dynamical Systems, 2015, 35 (11) : 5521-5553. doi: 10.3934/dcds.2015.35.5521 |
[3] |
Shi'an Wang, N. U. Ahmed. Optimal control and stabilization of building maintenance units based on minimum principle. Journal of Industrial & Management Optimization, 2021, 17 (4) : 1713-1727. doi: 10.3934/jimo.2020041 |
[4] |
Luke Finlay, Vladimir Gaitsgory, Ivan Lebedev. Linear programming solutions of periodic optimization problems: approximation of the optimal control. Journal of Industrial & Management Optimization, 2007, 3 (2) : 399-413. doi: 10.3934/jimo.2007.3.399 |
[5] |
Simone Cacace, Maurizio Falcone. A dynamic domain decomposition for the eikonal-diffusion equation. Discrete & Continuous Dynamical Systems - S, 2016, 9 (1) : 109-123. doi: 10.3934/dcdss.2016.9.109 |
[6] |
Livia Betz, Irwin Yousept. Optimal control of elliptic variational inequalities with bounded and unbounded operators. Mathematical Control & Related Fields, 2021 doi: 10.3934/mcrf.2021009 |
[7] |
Vladimir Gaitsgory, Ilya Shvartsman. Linear programming estimates for Cesàro and Abel limits of optimal values in optimal control problems. Discrete & Continuous Dynamical Systems - B, 2021 doi: 10.3934/dcdsb.2021102 |
[8] |
Fabio Camilli, Serikbolsyn Duisembay, Qing Tang. Approximation of an optimal control problem for the time-fractional Fokker-Planck equation. Journal of Dynamics & Games, 2021 doi: 10.3934/jdg.2021013 |
[9] |
Alexandr Mikhaylov, Victor Mikhaylov. Dynamic inverse problem for Jacobi matrices. Inverse Problems & Imaging, 2019, 13 (3) : 431-447. doi: 10.3934/ipi.2019021 |
[10] |
Shuting Chen, Zengji Du, Jiang Liu, Ke Wang. The dynamic properties of a generalized Kawahara equation with Kuramoto-Sivashinsky perturbation. Discrete & Continuous Dynamical Systems - B, 2021 doi: 10.3934/dcdsb.2021098 |
[11] |
Mohammed Abdelghany, Amr B. Eltawil, Zakaria Yahia, Kazuhide Nakata. A hybrid variable neighbourhood search and dynamic programming approach for the nurse rostering problem. Journal of Industrial & Management Optimization, 2021, 17 (4) : 2051-2072. doi: 10.3934/jimo.2020058 |
[12] |
Boris Kramer, John R. Singler. A POD projection method for large-scale algebraic Riccati equations. Numerical Algebra, Control & Optimization, 2016, 6 (4) : 413-435. doi: 10.3934/naco.2016018 |
[13] |
Ahmad El Hajj, Hassan Ibrahim, Vivian Rizik. $ BV $ solution for a non-linear Hamilton-Jacobi system. Discrete & Continuous Dynamical Systems, 2021, 41 (7) : 3273-3293. doi: 10.3934/dcds.2020405 |
[14] |
Olivier Ley, Erwin Topp, Miguel Yangari. Some results for the large time behavior of Hamilton-Jacobi equations with Caputo time derivative. Discrete & Continuous Dynamical Systems, 2021, 41 (8) : 3555-3577. doi: 10.3934/dcds.2021007 |
[15] |
Woocheol Choi, Youngwoo Koh. On the splitting method for the nonlinear Schrödinger equation with initial data in $ H^1 $. Discrete & Continuous Dynamical Systems, 2021, 41 (8) : 3837-3867. doi: 10.3934/dcds.2021019 |
[16] |
Amanda E. Diegel. A C0 interior penalty method for the Cahn-Hilliard equation. Electronic Research Archive, , () : -. doi: 10.3934/era.2021030 |
[17] |
Mohamed Ouzahra. Approximate controllability of the semilinear reaction-diffusion equation governed by a multiplicative control. Discrete & Continuous Dynamical Systems - B, 2021 doi: 10.3934/dcdsb.2021081 |
[18] |
Kehan Si, Zhenda Xu, Ka Fai Cedric Yiu, Xun Li. Open-loop solvability for mean-field stochastic linear quadratic optimal control problems of Markov regime-switching system. Journal of Industrial & Management Optimization, 2021 doi: 10.3934/jimo.2021074 |
[19] |
Ardeshir Ahmadi, Hamed Davari-Ardakani. A multistage stochastic programming framework for cardinality constrained portfolio optimization. Numerical Algebra, Control & Optimization, 2017, 7 (3) : 359-377. doi: 10.3934/naco.2017023 |
[20] |
Tobias Breiten, Sergey Dolgov, Martin Stoll. Solving differential Riccati equations: A nonlinear space-time method using tensor trains. Numerical Algebra, Control & Optimization, 2021, 11 (3) : 407-429. doi: 10.3934/naco.2020034 |
2019 Impact Factor: 0.857
Tools
Metrics
Other articles
by authors
[Back to Top]