Stochastic optimal control — A concise introduction

Jiongmin Yong

doi:10.3934/mcrf.2020027

Article Contents

2022, Volume 12, Issue 4: 1039-1136. Doi: 10.3934/mcrf.2020027

This issue Previous Article Control of reaction-diffusion models in biology and social sciences Next Article

Stochastic optimal control — A concise introduction

Jiongmin Yong

Department of Mathematics, University of Central Florida, Orlando, FL 32816, USA

Received: August 2019

Revised: January 2020

Early access: June 2020

Published: December 2022

This work is supported in part by NSF Grant DMS-1812921

Abstract Full Text(HTML) Related Papers Cited by

Abstract

This is a concise introduction to stochastic optimal control theory. We assume that the readers have basic knowledge of real analysis, functional analysis, elementary probability, ordinary differential equations and partial differential equations. We will present the following topics: (ⅰ) A brief presentation of relevant results on stochastic analysis; (ⅱ) Formulation of stochastic optimal control problems; (ⅲ) Variational method and Pontryagin's maximum principle, together with a brief introduction of backward stochastic differential equations; (ⅳ) Dynamic programming method and viscosity solutions to Hamilton-Jacobi-Bellman equation; (ⅴ) Linear-quadratic optimal control problems, including a careful discussion on open-loop optimal controls and closed-loop optimal strategies, linear forward-backward stochastic differential equations, and Riccati equations.

Keywords:

Mathematics Subject Classification: Primary: 93E20, 49K45, 49L05, 49L20, 49L25, 49N10; Secondary: 35D40, 35F21, 35Q93, 60H10.

Citation:

Full Text(HTML)

Related Papers

Cited by

References

[1]	M. G. Crandall and P. L. Lions, Viscosity solutions of Hamilton-Jacobi equations, Trans. Amer. Math. Soc., 277 (1983), 1-42. doi: 10.1090/S0002-9947-1983-0690039-8.
[2]	L. C. Evans and R. F. Gariepy, Measure Theory and Fine Properties of Functions, CRC Press, Boca Raton, FL, 1992.
[3]	W. H. Fleming and H. M. Soner, Controlled Markov Processes and Viscosity Solutions, Springer-Verlag, New York, 1993.
[4]	A. Gary, D. Greenhalgh, L. Hu, X. Mao and J. Pan, A stochastic differential equation SIS epidemic model, SIAM J. Appl. Math., 71 (2011), 876-902. doi: 10.1137/10081856X.
[5]	S. He, J. Wang and J. Yan, Semimartingale Theory and Stochastic Calculus, Science Press and CRC Press, Beijing, 1992.
[6]	I. Karatzas and S. Shreve, Brownian Motion and Stochastic Calculus, Springer-Verlag, New York, 1988, 47–127. doi: 10.1007/978-1-4684-0302-2_2.
[7]	E. Pardoux and S. Peng, Adapted solution of backward stochastic differential equations, Systems Control Lett., 14 (1990), 55-61. doi: 10.1016/0167-6911(90)90082-6.
[8]	S. Peng, A general stochastic maximum principle for optimal control problems, SIAM J. Control Optim., 28 (1990), 966-979. doi: 10.1137/0328054.
[9]	J. Sun, X. Li and J. Yong, Open-loop and closed-loop solvabilities for stochastic linear quadratic optimal control problems, SIAM J. Control Optim., 54 (2016), 2274-2308. doi: 10.1137/15M103532X.
[10]	J. Sun and J. Yong, Linear quadratic stochastic differential games: Open-loop and closed-loop saddle points, SIAM J. Control Optim., 52 (2014), 4082-4121. doi: 10.1137/140953642.
[11]	E. Tornatore, S. M. Buccellato and P. Vetro, Stability of a stochastic SIR system, Physica A, 354 (2005), 111-126. doi: 10.1016/j.physa.2005.02.057.
[12]	J. Yong and X. Y. Zhou, Stochastic Controls: Hamiltonian Systems and HJB Equations, Springer-Verlag, New York, 1999. doi: 10.1007/978-1-4612-1466-3.