American Institute of Mathematical Sciences

Advanced
January  2016, 1: 2 doi: 10.1186/s41546-016-0002-3

Mean-field stochastic linear quadratic optimal control problems: closed-loop solvability

Xun Li 1, , Jingrui Sun 1, and  Jiongmin Yong 2,,

1 Department of Applied Mathematics, The Hong Kong Polytechnic University, Hong Kong, China;

2 Department of Mathematics, University of Central Florida, Orlando, FL 32816, USA

Received  February 24, 2016 Revised  June 10, 2016

Fund Project: supported by Hong Kong RGC under grants 519913, 15209614 and 15224215. Jingrui Sun was partially supported by the National Natural Science Foundation of China (11401556) and the Fundamental Research Funds for the Central Universities (WK 2040000012). Jiongmin Yong was partially supported by NSF DMS-1406776.

An optimal control problem is studied for a linear mean-field stochastic differential equation with a quadratic cost functional. The coefficients and the weighting matrices in the cost functional are all assumed to be deterministic. Closedloop strategies are introduced, which require to be independent of initial states; and such a nature makes it very useful and convenient in applications. In this paper, the existence of an optimal closed-loop strategy for the system (also called the closedloop solvability of the problem) is characterized by the existence of a regular solution to the coupled two (generalized) Riccati equations, together with some constraints on the adapted solution to a linear backward stochastic differential equation and a linear terminal value problem of an ordinary differential equation.
Keywords: Mean-field stochastic differential equation, Linear quadratic optimal control, Riccati equation, Regular solution, Closed-loop solvability.
Mathematics Subject Classification: 49N10;49N35;93E20.
Citation: Xun Li, Jingrui Sun, Jiongmin Yong. Mean-field stochastic linear quadratic optimal control problems: closed-loop solvability. Probability, Uncertainty and Quantitative Risk, doi: 10.1186/s41546-016-0002-3
References:
[1]

Ait Rami, M, Moore, JB, Zhou, XY:Indefinite stochastic linear quadratic control and generalized differential Riccati equation. SIAM J. Control Optim 40, 1296-1311 (2001) Google Scholar

[2]

Andersson, D, Djehiche, B:A maximum principle for SDEs of mean-field type. Appl. Math. Optim 63, 341-356 (2011) Google Scholar

[3]

Athans, M:The matrix minimum principle. Inform. Control 11, 592-606 (1968) Google Scholar

[4]

Buckdahn, R, Djehiche, B, Li, J:A general stochastic maximum principle for SDEs of mean-field type.Appl. Math. Optim 64, 197-216 (2011) Google Scholar

[5]

Buckdahn, R, Djehiche, B, Li, J, Peng, S:Mean-field backward stochastic differential equations:a limit approach. Ann. Probab 37, 1524-1565 (2009) Google Scholar

[6]

Buckdahn, R, Li, J, Peng, S:Mean-field backward stochastic differential equations and related partial differential equations. Stoch. Proc. Appl 119, 3133-3154 (2009) Google Scholar

[7]

Chen, S, Li, X, Zhou, XY:Stochastic linear quadratic regulators with indefinite control weight costs.SIAM J. Control Optim 36, 1685-1702 (1998) Google Scholar

[8]

Chen, S, Yong, J:Stochastic linear quadratic optimal control problems with random coefficients. Chin.Ann. Math 21B, 323-338 (2000) Google Scholar

[9]

Cui, XY, Li, X, Li, D:Unified framework of mean-field formulations for optimal multi-period meanvariance portfolio selection. IEEE Trans. Auto. Control 59, 1833-1844 (2014) Google Scholar

[10]

Elliott, R, Li, X, Ni, YH:Discrete time mean-field stochastic linear-quadratic optimal control problems.Automatica 49, 3222-3233 (2013) Google Scholar

[11]

Huang, J, Li, X, Wang, TX:Mean-field linear-quadratic-Gaussian (LQG) games for stochastic integral systems. IEEE Trans. Auto. Control (2015). doi:10.1109/TAC.2015.2506620 Google Scholar

[12]

Huang, J, Li, X, Yong, J:A linear-quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon. Math. Control Related Fields 5, 97-139 (2015) Google Scholar

[13]

Kac, M:Foundations of kinetic theory. Proc. Third Berkeley Symp. Math. Stat. Probab 3, 171-197 (1956) Google Scholar

[14]

McKean, HP:A class of Markov processes associated with nonlinear parabolic equations. Proc. Natl.Acad. Sci. USA 56, 1907-1911 (1966) Google Scholar

[15]

Meyer-Brandis, T, Øksendal, B, Zhou, XY:A mean-field stochastic maximum principle via Malliavin calculus. Stochastics 84(5-6), 643-666 (2012). doi:10.1080/17442508.2011.651619 Google Scholar

[16]

Penrose, R:A generalized inverse of matrices. Proc. Cambridge Philos Soc 52, 17-19 (1955) Google Scholar

[17]

Sun, J:Mean-field stochastic linear quadratic optimal control problems:open-loop solvabilities. ESAIM:COCV, 016023 (2016). doi:10.1051/cocv/2 Google Scholar

[18]

Sun, J, Yong, J:Linear quadratic stochastic differential games:open-loop and closed-loop saddle points.SIAM J. Control Optim 52, 4082-4121 (2014) Google Scholar

[19]

Sun, J, Yong, J, Zhang, S:Linear quadratic stochastic two-person zero-sum differential games in an infinite horizon. ESAIM COCV 22, 743-769 (2016). doi:10.1051/cocv/2015024 Google Scholar

[20]

Tang, S:General linear quadratic optimal stochastic control problems with random coefficients:linear stochastic Hamilton systems and backward stochastic Riccati equations. SIAM J. Control Optim 42, 53-75 (2003) Google Scholar

[21]

Wonham, WM:On a matrix Riccati equation of stochastic control. SIAM J. Control Optim 6, 681-697(1968) Google Scholar

[22]

Yong, J:Linear-quadratic optimal control problems for mean-field stochastic differential equations. SIAM J. Control Optim 51, 2809-2838 (2013) Google Scholar

[23]

Yong, J, Zhou, XY:Stochastic controls:Hamiltonian systems and HJB equations. Springer-Verlag, New York (1999) Google Scholar

show all references

References:
[1]

Ait Rami, M, Moore, JB, Zhou, XY:Indefinite stochastic linear quadratic control and generalized differential Riccati equation. SIAM J. Control Optim 40, 1296-1311 (2001) Google Scholar

[2]

Andersson, D, Djehiche, B:A maximum principle for SDEs of mean-field type. Appl. Math. Optim 63, 341-356 (2011) Google Scholar

[3]

Athans, M:The matrix minimum principle. Inform. Control 11, 592-606 (1968) Google Scholar

[4]

Buckdahn, R, Djehiche, B, Li, J:A general stochastic maximum principle for SDEs of mean-field type.Appl. Math. Optim 64, 197-216 (2011) Google Scholar

[5]

Buckdahn, R, Djehiche, B, Li, J, Peng, S:Mean-field backward stochastic differential equations:a limit approach. Ann. Probab 37, 1524-1565 (2009) Google Scholar

[6]

Buckdahn, R, Li, J, Peng, S:Mean-field backward stochastic differential equations and related partial differential equations. Stoch. Proc. Appl 119, 3133-3154 (2009) Google Scholar

[7]

Chen, S, Li, X, Zhou, XY:Stochastic linear quadratic regulators with indefinite control weight costs.SIAM J. Control Optim 36, 1685-1702 (1998) Google Scholar

[8]

Chen, S, Yong, J:Stochastic linear quadratic optimal control problems with random coefficients. Chin.Ann. Math 21B, 323-338 (2000) Google Scholar

[9]

Cui, XY, Li, X, Li, D:Unified framework of mean-field formulations for optimal multi-period meanvariance portfolio selection. IEEE Trans. Auto. Control 59, 1833-1844 (2014) Google Scholar

[10]

Elliott, R, Li, X, Ni, YH:Discrete time mean-field stochastic linear-quadratic optimal control problems.Automatica 49, 3222-3233 (2013) Google Scholar

[11]

Huang, J, Li, X, Wang, TX:Mean-field linear-quadratic-Gaussian (LQG) games for stochastic integral systems. IEEE Trans. Auto. Control (2015). doi:10.1109/TAC.2015.2506620 Google Scholar

[12]

Huang, J, Li, X, Yong, J:A linear-quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon. Math. Control Related Fields 5, 97-139 (2015) Google Scholar

[13]

Kac, M:Foundations of kinetic theory. Proc. Third Berkeley Symp. Math. Stat. Probab 3, 171-197 (1956) Google Scholar

[14]

McKean, HP:A class of Markov processes associated with nonlinear parabolic equations. Proc. Natl.Acad. Sci. USA 56, 1907-1911 (1966) Google Scholar

[15]

Meyer-Brandis, T, Øksendal, B, Zhou, XY:A mean-field stochastic maximum principle via Malliavin calculus. Stochastics 84(5-6), 643-666 (2012). doi:10.1080/17442508.2011.651619 Google Scholar

[16]

Penrose, R:A generalized inverse of matrices. Proc. Cambridge Philos Soc 52, 17-19 (1955) Google Scholar

[17]

Sun, J:Mean-field stochastic linear quadratic optimal control problems:open-loop solvabilities. ESAIM:COCV, 016023 (2016). doi:10.1051/cocv/2 Google Scholar

[18]

Sun, J, Yong, J:Linear quadratic stochastic differential games:open-loop and closed-loop saddle points.SIAM J. Control Optim 52, 4082-4121 (2014) Google Scholar

[19]

Sun, J, Yong, J, Zhang, S:Linear quadratic stochastic two-person zero-sum differential games in an infinite horizon. ESAIM COCV 22, 743-769 (2016). doi:10.1051/cocv/2015024 Google Scholar

[20]

Tang, S:General linear quadratic optimal stochastic control problems with random coefficients:linear stochastic Hamilton systems and backward stochastic Riccati equations. SIAM J. Control Optim 42, 53-75 (2003) Google Scholar

[21]

Wonham, WM:On a matrix Riccati equation of stochastic control. SIAM J. Control Optim 6, 681-697(1968) Google Scholar

[22]

Yong, J:Linear-quadratic optimal control problems for mean-field stochastic differential equations. SIAM J. Control Optim 51, 2809-2838 (2013) Google Scholar

[23]

Yong, J, Zhou, XY:Stochastic controls:Hamiltonian systems and HJB equations. Springer-Verlag, New York (1999) Google Scholar

[1]

Kehan Si, Zhenda Xu, Ka Fai Cedric Yiu, Xun Li. Open-loop solvability for mean-field stochastic linear quadratic optimal control problems of Markov regime-switching system. Journal of Industrial & Management Optimization, 2021  doi: 10.3934/jimo.2021074
[2]

Jun Moon. Linear-quadratic mean-field type stackelberg differential games for stochastic jump-diffusion systems. Mathematical Control & Related Fields, 2021  doi: 10.3934/mcrf.2021026
[3]

Diana Keller. Optimal control of a linear stochastic Schrödinger equation. Conference Publications, doi: 10.3934/proc.2013.2013.437
[4]

Marco Cirant, Diogo A. Gomes, Edgard A. Pimentel, Héctor Sánchez-Morgado. On some singular mean-field games. Journal of Dynamics & Games, 2021  doi: 10.3934/jdg.2021006
[5]

Seung-Yeal Ha, Jinwook Jung, Jeongho Kim, Jinyeong Park, Xiongtao Zhang. A mean-field limit of the particle swarmalator model. Kinetic & Related Models, doi: 10.3934/krm.2021011
[6]

Benrong Zheng, Xianpei Hong. Effects of take-back legislation on pricing and coordination in a closed-loop supply chain. Journal of Industrial & Management Optimization, 2021  doi: 10.3934/jimo.2021035
[7]

Reza Lotfi, Yahia Zare Mehrjerdi, Mir Saman Pishvaee, Ahmad Sadeghieh, Gerhard-Wilhelm Weber. A robust optimization model for sustainable and resilient closed-loop supply chain network design considering conditional value at risk. Numerical Algebra, Control & Optimization, doi: 10.3934/naco.2020023
[8]

René Aïd, Roxana Dumitrescu, Peter Tankov. The entry and exit game in the electricity markets: A mean-field game approach. Journal of Dynamics & Games, 2021  doi: 10.3934/jdg.2021012
[9]

Shanjian Tang, Fu Zhang. Path-dependent optimal stochastic control and viscosity solution of associated Bellman equations. Discrete & Continuous Dynamical Systems, doi: 10.3934/dcds.2015.35.5521
[10]

Changpin Li, Zhiqiang Li. Asymptotic behaviors of solution to partial differential equation with Caputo–Hadamard derivative and fractional Laplacian: Hyperbolic case. Discrete & Continuous Dynamical Systems - S, 2021  doi: 10.3934/dcdss.2021023
[11]

Julian Tugaut. Captivity of the solution to the granular media equation. Kinetic & Related Models, doi: 10.3934/krm.2021002
[12]

Zhiming Guo, Zhi-Chun Yang, Xingfu Zou. Existence and uniqueness of positive solution to a non-local differential equation with homogeneous Dirichlet boundary condition---A non-monotone case. Communications on Pure & Applied Analysis, doi: 10.3934/cpaa.2012.11.1825
[13]

Nhu N. Nguyen, George Yin. Stochastic partial differential equation models for spatially dependent predator-prey equations. Discrete & Continuous Dynamical Systems - B, doi: 10.3934/dcdsb.2019175
[14]

Seddigheh Banihashemi, Hossein Jafaria, Afshin Babaei. A novel collocation approach to solve a nonlinear stochastic differential equation of fractional order involving a constant delay. Discrete & Continuous Dynamical Systems - S, 2021  doi: 10.3934/dcdss.2021025
[15]

Fabio Camilli, Serikbolsyn Duisembay, Qing Tang. Approximation of an optimal control problem for the time-fractional Fokker-Planck equation. Journal of Dynamics & Games, 2021  doi: 10.3934/jdg.2021013
[16]

Yingxu Tian, Junyi Guo, Zhongyang Sun. Optimal mean-variance reinsurance in a financial market with stochastic rate of return. Journal of Industrial & Management Optimization, doi: 10.3934/jimo.2020051
[17]

Luke Finlay, Vladimir Gaitsgory, Ivan Lebedev. Linear programming solutions of periodic optimization problems: approximation of the optimal control. Journal of Industrial & Management Optimization, doi: 10.3934/jimo.2007.3.399
[18]

Qi Lü, Xu Zhang. A concise introduction to control theory for stochastic partial differential equations. Mathematical Control & Related Fields, 2021  doi: 10.3934/mcrf.2021020
[19]

Meng-Xue Chang, Bang-Sheng Han, Xiao-Ming Fan. Global dynamics of the solution for a bistable reaction diffusion equation with nonlocal effect. Electronic Research Archive, doi: 10.3934/era.2021024
[20]

Vladimir Gaitsgory, Ilya Shvartsman. Linear programming estimates for Cesàro and Abel limits of optimal values in optimal control problems. Discrete & Continuous Dynamical Systems - B, 2021  doi: 10.3934/dcdsb.2021102

 Impact Factor: 

Tools

Metrics

  • PDF downloads (10)
  • HTML views (0)
  • Cited by (13)

Other articles
by authors

[Back to Top]

Copyright © 2021 American Institute of Mathematical Sciences