-
Previous Article
Time-consistent lifetime portfolio selection under smooth ambiguity
- MCRF Home
- This Issue
-
Next Article
Continuity with respect to the speed for optimal ship forms based on Michell's formula
Online First articles are published articles within a journal that have not yet been assigned to a formal issue. This means they do not yet have a volume number, issue number, or page numbers assigned to them, however, they can still be found and cited using their DOI (Digital Object Identifier). Online First publication benefits the research community by making new scientific discoveries known as quickly as possible.
Readers can access Online First articles via the “Online First” tab for the selected journal.
Stochastic optimal control — A concise introduction
Department of Mathematics, University of Central Florida, Orlando, FL 32816, USA |
This is a concise introduction to stochastic optimal control theory. We assume that the readers have basic knowledge of real analysis, functional analysis, elementary probability, ordinary differential equations and partial differential equations. We will present the following topics: (ⅰ) A brief presentation of relevant results on stochastic analysis; (ⅱ) Formulation of stochastic optimal control problems; (ⅲ) Variational method and Pontryagin's maximum principle, together with a brief introduction of backward stochastic differential equations; (ⅳ) Dynamic programming method and viscosity solutions to Hamilton-Jacobi-Bellman equation; (ⅴ) Linear-quadratic optimal control problems, including a careful discussion on open-loop optimal controls and closed-loop optimal strategies, linear forward-backward stochastic differential equations, and Riccati equations.
References:
| [1] |
M. G. Crandall and P. L. Lions,
Viscosity solutions of Hamilton-Jacobi equations, Trans. Amer. Math. Soc., 277 (1983), 1-42.
doi: 10.1090/S0002-9947-1983-0690039-8. |
| [2] |
L. C. Evans and R. F. Gariepy, Measure Theory and Fine Properties of Functions, CRC Press, Boca Raton, FL, 1992.
|
| [3] |
W. H. Fleming and H. M. Soner, Controlled Markov Processes and Viscosity Solutions, Springer-Verlag, New York, 1993. |
| [4] |
A. Gary, D. Greenhalgh, L. Hu, X. Mao and J. Pan,
A stochastic differential equation SIS epidemic model, SIAM J. Appl. Math., 71 (2011), 876-902.
doi: 10.1137/10081856X. |
| [5] |
S. He, J. Wang and J. Yan, Semimartingale Theory and Stochastic Calculus, Science Press and CRC Press, Beijing, 1992.
|
| [6] |
I. Karatzas and S. Shreve, Brownian Motion and Stochastic Calculus, Springer-Verlag, New York, 1988, 47–127.
doi: 10.1007/978-1-4684-0302-2_2. |
| [7] |
E. Pardoux and S. Peng,
Adapted solution of backward stochastic differential equations, Systems Control Lett., 14 (1990), 55-61.
doi: 10.1016/0167-6911(90)90082-6. |
| [8] |
S. Peng,
A general stochastic maximum principle for optimal control problems, SIAM J. Control Optim., 28 (1990), 966-979.
doi: 10.1137/0328054. |
| [9] |
J. Sun, X. Li and J. Yong,
Open-loop and closed-loop solvabilities for stochastic linear quadratic optimal control problems, SIAM J. Control Optim., 54 (2016), 2274-2308.
doi: 10.1137/15M103532X. |
| [10] |
J. Sun and J. Yong,
Linear quadratic stochastic differential games: Open-loop and closed-loop saddle points, SIAM J. Control Optim., 52 (2014), 4082-4121.
doi: 10.1137/140953642. |
| [11] |
E. Tornatore, S. M. Buccellato and P. Vetro,
Stability of a stochastic SIR system, Physica A, 354 (2005), 111-126.
doi: 10.1016/j.physa.2005.02.057. |
| [12] |
J. Yong and X. Y. Zhou, Stochastic Controls: Hamiltonian Systems and HJB Equations, Springer-Verlag, New York, 1999.
doi: 10.1007/978-1-4612-1466-3. |
show all references
References:
| [1] |
M. G. Crandall and P. L. Lions,
Viscosity solutions of Hamilton-Jacobi equations, Trans. Amer. Math. Soc., 277 (1983), 1-42.
doi: 10.1090/S0002-9947-1983-0690039-8. |
| [2] |
L. C. Evans and R. F. Gariepy, Measure Theory and Fine Properties of Functions, CRC Press, Boca Raton, FL, 1992.
|
| [3] |
W. H. Fleming and H. M. Soner, Controlled Markov Processes and Viscosity Solutions, Springer-Verlag, New York, 1993. |
| [4] |
A. Gary, D. Greenhalgh, L. Hu, X. Mao and J. Pan,
A stochastic differential equation SIS epidemic model, SIAM J. Appl. Math., 71 (2011), 876-902.
doi: 10.1137/10081856X. |
| [5] |
S. He, J. Wang and J. Yan, Semimartingale Theory and Stochastic Calculus, Science Press and CRC Press, Beijing, 1992.
|
| [6] |
I. Karatzas and S. Shreve, Brownian Motion and Stochastic Calculus, Springer-Verlag, New York, 1988, 47–127.
doi: 10.1007/978-1-4684-0302-2_2. |
| [7] |
E. Pardoux and S. Peng,
Adapted solution of backward stochastic differential equations, Systems Control Lett., 14 (1990), 55-61.
doi: 10.1016/0167-6911(90)90082-6. |
| [8] |
S. Peng,
A general stochastic maximum principle for optimal control problems, SIAM J. Control Optim., 28 (1990), 966-979.
doi: 10.1137/0328054. |
| [9] |
J. Sun, X. Li and J. Yong,
Open-loop and closed-loop solvabilities for stochastic linear quadratic optimal control problems, SIAM J. Control Optim., 54 (2016), 2274-2308.
doi: 10.1137/15M103532X. |
| [10] |
J. Sun and J. Yong,
Linear quadratic stochastic differential games: Open-loop and closed-loop saddle points, SIAM J. Control Optim., 52 (2014), 4082-4121.
doi: 10.1137/140953642. |
| [11] |
E. Tornatore, S. M. Buccellato and P. Vetro,
Stability of a stochastic SIR system, Physica A, 354 (2005), 111-126.
doi: 10.1016/j.physa.2005.02.057. |
| [12] |
J. Yong and X. Y. Zhou, Stochastic Controls: Hamiltonian Systems and HJB Equations, Springer-Verlag, New York, 1999.
doi: 10.1007/978-1-4612-1466-3. |
| [1] |
Xuhui Wang, Lei Hu. A new method to solve the Hamilton-Jacobi-Bellman equation for a stochastic portfolio optimization model with boundary memory. Journal of Industrial and Management Optimization, 2021 doi: 10.3934/jimo.2021137 |
| [2] |
Steven Richardson, Song Wang. The viscosity approximation to the Hamilton-Jacobi-Bellman equation in optimal feedback control: Upper bounds for extended domains. Journal of Industrial and Management Optimization, 2010, 6 (1) : 161-175. doi: 10.3934/jimo.2010.6.161 |
| [3] |
Zhen-Zhen Tao, Bing Sun. A feedback design for numerical solution to optimal control problems based on Hamilton-Jacobi-Bellman equation. Electronic Research Archive, 2021, 29 (5) : 3429-3447. doi: 10.3934/era.2021046 |
| [4] |
Jean-Claude Zambrini. On the geometry of the Hamilton-Jacobi-Bellman equation. Journal of Geometric Mechanics, 2009, 1 (3) : 369-387. doi: 10.3934/jgm.2009.1.369 |
| [5] |
Bian-Xia Yang, Shanshan Gu, Guowei Dai. Existence and multiplicity for Hamilton-Jacobi-Bellman equation. Communications on Pure and Applied Analysis, 2021, 20 (11) : 3767-3793. doi: 10.3934/cpaa.2021130 |
| [6] |
Daniele Castorina, Annalisa Cesaroni, Luca Rossi. On a parabolic Hamilton-Jacobi-Bellman equation degenerating at the boundary. Communications on Pure and Applied Analysis, 2016, 15 (4) : 1251-1263. doi: 10.3934/cpaa.2016.15.1251 |
| [7] |
Guy Barles, Ariela Briani, Emmanuel Trélat. Value function for regional control problems via dynamic programming and Pontryagin maximum principle. Mathematical Control and Related Fields, 2018, 8 (3&4) : 509-533. doi: 10.3934/mcrf.2018021 |
| [8] |
Joan-Andreu Lázaro-Camí, Juan-Pablo Ortega. The stochastic Hamilton-Jacobi equation. Journal of Geometric Mechanics, 2009, 1 (3) : 295-315. doi: 10.3934/jgm.2009.1.295 |
| [9] |
Stefan Doboszczak, Manil T. Mohan, Sivaguru S. Sritharan. Pontryagin maximum principle for the optimal control of linearized compressible navier-stokes equations with state constraints. Evolution Equations and Control Theory, 2022, 11 (2) : 347-371. doi: 10.3934/eect.2020110 |
| [10] |
Nicolas Forcadel, Mamdouh Zaydan. A comparison principle for Hamilton-Jacobi equation with moving in time boundary. Evolution Equations and Control Theory, 2019, 8 (3) : 543-565. doi: 10.3934/eect.2019026 |
| [11] |
Mohamed Assellaou, Olivier Bokanowski, Hasnaa Zidani. Error estimates for second order Hamilton-Jacobi-Bellman equations. Approximation of probabilistic reachable sets. Discrete and Continuous Dynamical Systems, 2015, 35 (9) : 3933-3964. doi: 10.3934/dcds.2015.35.3933 |
| [12] |
Zhen Wu, Feng Zhang. Maximum principle for discrete-time stochastic optimal control problem and stochastic game. Mathematical Control and Related Fields, 2022, 12 (2) : 475-493. doi: 10.3934/mcrf.2021031 |
| [13] |
Tomoki Ohsawa, Anthony M. Bloch. Nonholonomic Hamilton-Jacobi equation and integrability. Journal of Geometric Mechanics, 2009, 1 (4) : 461-481. doi: 10.3934/jgm.2009.1.461 |
| [14] |
Nalini Anantharaman, Renato Iturriaga, Pablo Padilla, Héctor Sánchez-Morgado. Physical solutions of the Hamilton-Jacobi equation. Discrete and Continuous Dynamical Systems - B, 2005, 5 (3) : 513-528. doi: 10.3934/dcdsb.2005.5.513 |
| [15] |
María Barbero-Liñán, Manuel de León, David Martín de Diego, Juan C. Marrero, Miguel C. Muñoz-Lecanda. Kinematic reduction and the Hamilton-Jacobi equation. Journal of Geometric Mechanics, 2012, 4 (3) : 207-237. doi: 10.3934/jgm.2012.4.207 |
| [16] |
Larry M. Bates, Francesco Fassò, Nicola Sansonetto. The Hamilton-Jacobi equation, integrability, and nonholonomic systems. Journal of Geometric Mechanics, 2014, 6 (4) : 441-449. doi: 10.3934/jgm.2014.6.441 |
| [17] |
Xiao-Li Ding, Iván Area, Juan J. Nieto. Controlled singular evolution equations and Pontryagin type maximum principle with applications. Evolution Equations and Control Theory, 2021 doi: 10.3934/eect.2021059 |
| [18] |
Yan Wang, Yanxiang Zhao, Lei Wang, Aimin Song, Yanping Ma. Stochastic maximum principle for partial information optimal investment and dividend problem of an insurer. Journal of Industrial and Management Optimization, 2018, 14 (2) : 653-671. doi: 10.3934/jimo.2017067 |
| [19] |
Shanjian Tang. A second-order maximum principle for singular optimal stochastic controls. Discrete and Continuous Dynamical Systems - B, 2010, 14 (4) : 1581-1599. doi: 10.3934/dcdsb.2010.14.1581 |
| [20] |
Shaolin Ji, Xiaole Xue. A stochastic maximum principle for linear quadratic problem with nonconvex control domain. Mathematical Control and Related Fields, 2019, 9 (3) : 495-507. doi: 10.3934/mcrf.2019022 |
2020 Impact Factor: 1.284
Tools
Metrics
Other articles
by authors
[Back to Top]