American Institute of Mathematical Sciences

Advanced
January  2016, 1: 2 doi: 10.1186/s41546-016-0002-3

Mean-field stochastic linear quadratic optimal control problems: closed-loop solvability

Xun Li 1, , Jingrui Sun 1, and  Jiongmin Yong 2,,

1 Department of Applied Mathematics, The Hong Kong Polytechnic University, Hong Kong, China;

2 Department of Mathematics, University of Central Florida, Orlando, FL 32816, USA

Received  February 24, 2016 Revised  June 10, 2016 Published  August 2016

Fund Project: supported by Hong Kong RGC under grants 519913, 15209614 and 15224215. Jingrui Sun was partially supported by the National Natural Science Foundation of China (11401556) and the Fundamental Research Funds for the Central Universities (WK 2040000012). Jiongmin Yong was partially supported by NSF DMS-1406776.

An optimal control problem is studied for a linear mean-field stochastic differential equation with a quadratic cost functional. The coefficients and the weighting matrices in the cost functional are all assumed to be deterministic. Closedloop strategies are introduced, which require to be independent of initial states; and such a nature makes it very useful and convenient in applications. In this paper, the existence of an optimal closed-loop strategy for the system (also called the closedloop solvability of the problem) is characterized by the existence of a regular solution to the coupled two (generalized) Riccati equations, together with some constraints on the adapted solution to a linear backward stochastic differential equation and a linear terminal value problem of an ordinary differential equation.
Keywords: Mean-field stochastic differential equation, Linear quadratic optimal control, Riccati equation, Regular solution, Closed-loop solvability.
Mathematics Subject Classification: 49N10;49N35;93E20.
Citation: Xun Li, Jingrui Sun, Jiongmin Yong. Mean-field stochastic linear quadratic optimal control problems: closed-loop solvability. Probability, Uncertainty and Quantitative Risk, 2016, 1 (0) : 2-. doi: 10.1186/s41546-016-0002-3
References:
[1]

Ait Rami, M, Moore, JB, Zhou, XY:Indefinite stochastic linear quadratic control and generalized differential Riccati equation. SIAM J. Control Optim 40, 1296-1311 (2001) Google Scholar

[2]

Andersson, D, Djehiche, B:A maximum principle for SDEs of mean-field type. Appl. Math. Optim 63, 341-356 (2011) Google Scholar

[3]

Athans, M:The matrix minimum principle. Inform. Control 11, 592-606 (1968) Google Scholar

[4]

Buckdahn, R, Djehiche, B, Li, J:A general stochastic maximum principle for SDEs of mean-field type.Appl. Math. Optim 64, 197-216 (2011) Google Scholar

[5]

Buckdahn, R, Djehiche, B, Li, J, Peng, S:Mean-field backward stochastic differential equations:a limit approach. Ann. Probab 37, 1524-1565 (2009) Google Scholar

[6]

Buckdahn, R, Li, J, Peng, S:Mean-field backward stochastic differential equations and related partial differential equations. Stoch. Proc. Appl 119, 3133-3154 (2009) Google Scholar

[7]

Chen, S, Li, X, Zhou, XY:Stochastic linear quadratic regulators with indefinite control weight costs.SIAM J. Control Optim 36, 1685-1702 (1998) Google Scholar

[8]

Chen, S, Yong, J:Stochastic linear quadratic optimal control problems with random coefficients. Chin.Ann. Math 21B, 323-338 (2000) Google Scholar

[9]

Cui, XY, Li, X, Li, D:Unified framework of mean-field formulations for optimal multi-period meanvariance portfolio selection. IEEE Trans. Auto. Control 59, 1833-1844 (2014) Google Scholar

[10]

Elliott, R, Li, X, Ni, YH:Discrete time mean-field stochastic linear-quadratic optimal control problems.Automatica 49, 3222-3233 (2013) Google Scholar

[11]

Huang, J, Li, X, Wang, TX:Mean-field linear-quadratic-Gaussian (LQG) games for stochastic integral systems. IEEE Trans. Auto. Control (2015). doi:10.1109/TAC.2015.2506620 Google Scholar

[12]

Huang, J, Li, X, Yong, J:A linear-quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon. Math. Control Related Fields 5, 97-139 (2015) Google Scholar

[13]

Kac, M:Foundations of kinetic theory. Proc. Third Berkeley Symp. Math. Stat. Probab 3, 171-197 (1956) Google Scholar

[14]

McKean, HP:A class of Markov processes associated with nonlinear parabolic equations. Proc. Natl.Acad. Sci. USA 56, 1907-1911 (1966) Google Scholar

[15]

Meyer-Brandis, T, Øksendal, B, Zhou, XY:A mean-field stochastic maximum principle via Malliavin calculus. Stochastics 84(5-6), 643-666 (2012). doi:10.1080/17442508.2011.651619 Google Scholar

[16]

Penrose, R:A generalized inverse of matrices. Proc. Cambridge Philos Soc 52, 17-19 (1955) Google Scholar

[17]

Sun, J:Mean-field stochastic linear quadratic optimal control problems:open-loop solvabilities. ESAIM:COCV, 016023 (2016). doi:10.1051/cocv/2 Google Scholar

[18]

Sun, J, Yong, J:Linear quadratic stochastic differential games:open-loop and closed-loop saddle points.SIAM J. Control Optim 52, 4082-4121 (2014) Google Scholar

[19]

Sun, J, Yong, J, Zhang, S:Linear quadratic stochastic two-person zero-sum differential games in an infinite horizon. ESAIM COCV 22, 743-769 (2016). doi:10.1051/cocv/2015024 Google Scholar

[20]

Tang, S:General linear quadratic optimal stochastic control problems with random coefficients:linear stochastic Hamilton systems and backward stochastic Riccati equations. SIAM J. Control Optim 42, 53-75 (2003) Google Scholar

[21]

Wonham, WM:On a matrix Riccati equation of stochastic control. SIAM J. Control Optim 6, 681-697(1968) Google Scholar

[22]

Yong, J:Linear-quadratic optimal control problems for mean-field stochastic differential equations. SIAM J. Control Optim 51, 2809-2838 (2013) Google Scholar

[23]

Yong, J, Zhou, XY:Stochastic controls:Hamiltonian systems and HJB equations. Springer-Verlag, New York (1999) Google Scholar

show all references

References:
[1]

Ait Rami, M, Moore, JB, Zhou, XY:Indefinite stochastic linear quadratic control and generalized differential Riccati equation. SIAM J. Control Optim 40, 1296-1311 (2001) Google Scholar

[2]

Andersson, D, Djehiche, B:A maximum principle for SDEs of mean-field type. Appl. Math. Optim 63, 341-356 (2011) Google Scholar

[3]

Athans, M:The matrix minimum principle. Inform. Control 11, 592-606 (1968) Google Scholar

[4]

Buckdahn, R, Djehiche, B, Li, J:A general stochastic maximum principle for SDEs of mean-field type.Appl. Math. Optim 64, 197-216 (2011) Google Scholar

[5]

Buckdahn, R, Djehiche, B, Li, J, Peng, S:Mean-field backward stochastic differential equations:a limit approach. Ann. Probab 37, 1524-1565 (2009) Google Scholar

[6]

Buckdahn, R, Li, J, Peng, S:Mean-field backward stochastic differential equations and related partial differential equations. Stoch. Proc. Appl 119, 3133-3154 (2009) Google Scholar

[7]

Chen, S, Li, X, Zhou, XY:Stochastic linear quadratic regulators with indefinite control weight costs.SIAM J. Control Optim 36, 1685-1702 (1998) Google Scholar

[8]

Chen, S, Yong, J:Stochastic linear quadratic optimal control problems with random coefficients. Chin.Ann. Math 21B, 323-338 (2000) Google Scholar

[9]

Cui, XY, Li, X, Li, D:Unified framework of mean-field formulations for optimal multi-period meanvariance portfolio selection. IEEE Trans. Auto. Control 59, 1833-1844 (2014) Google Scholar

[10]

Elliott, R, Li, X, Ni, YH:Discrete time mean-field stochastic linear-quadratic optimal control problems.Automatica 49, 3222-3233 (2013) Google Scholar

[11]

Huang, J, Li, X, Wang, TX:Mean-field linear-quadratic-Gaussian (LQG) games for stochastic integral systems. IEEE Trans. Auto. Control (2015). doi:10.1109/TAC.2015.2506620 Google Scholar

[12]

Huang, J, Li, X, Yong, J:A linear-quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon. Math. Control Related Fields 5, 97-139 (2015) Google Scholar

[13]

Kac, M:Foundations of kinetic theory. Proc. Third Berkeley Symp. Math. Stat. Probab 3, 171-197 (1956) Google Scholar

[14]

McKean, HP:A class of Markov processes associated with nonlinear parabolic equations. Proc. Natl.Acad. Sci. USA 56, 1907-1911 (1966) Google Scholar

[15]

Meyer-Brandis, T, Øksendal, B, Zhou, XY:A mean-field stochastic maximum principle via Malliavin calculus. Stochastics 84(5-6), 643-666 (2012). doi:10.1080/17442508.2011.651619 Google Scholar

[16]

Penrose, R:A generalized inverse of matrices. Proc. Cambridge Philos Soc 52, 17-19 (1955) Google Scholar

[17]

Sun, J:Mean-field stochastic linear quadratic optimal control problems:open-loop solvabilities. ESAIM:COCV, 016023 (2016). doi:10.1051/cocv/2 Google Scholar

[18]

Sun, J, Yong, J:Linear quadratic stochastic differential games:open-loop and closed-loop saddle points.SIAM J. Control Optim 52, 4082-4121 (2014) Google Scholar

[19]

Sun, J, Yong, J, Zhang, S:Linear quadratic stochastic two-person zero-sum differential games in an infinite horizon. ESAIM COCV 22, 743-769 (2016). doi:10.1051/cocv/2015024 Google Scholar

[20]

Tang, S:General linear quadratic optimal stochastic control problems with random coefficients:linear stochastic Hamilton systems and backward stochastic Riccati equations. SIAM J. Control Optim 42, 53-75 (2003) Google Scholar

[21]

Wonham, WM:On a matrix Riccati equation of stochastic control. SIAM J. Control Optim 6, 681-697(1968) Google Scholar

[22]

Yong, J:Linear-quadratic optimal control problems for mean-field stochastic differential equations. SIAM J. Control Optim 51, 2809-2838 (2013) Google Scholar

[23]

Yong, J, Zhou, XY:Stochastic controls:Hamiltonian systems and HJB equations. Springer-Verlag, New York (1999) Google Scholar

[1]

Jingrui Sun, Hanxiao Wang. Mean-field stochastic linear-quadratic optimal control problems: Weak closed-loop solvability. Mathematical Control & Related Fields, 2021, 11 (1) : 47-71. doi: 10.3934/mcrf.2020026
[2]

Hanxiao Wang, Jingrui Sun, Jiongmin Yong. Weak closed-loop solvability of stochastic linear-quadratic optimal control problems. Discrete & Continuous Dynamical Systems, 2019, 39 (5) : 2785-2805. doi: 10.3934/dcds.2019117
[3]

Kehan Si, Zhenda Xu, Ka Fai Cedric Yiu, Xun Li. Open-loop solvability for mean-field stochastic linear quadratic optimal control problems of Markov regime-switching system. Journal of Industrial & Management Optimization, 2021  doi: 10.3934/jimo.2021074
[4]

Jianhui Huang, Xun Li, Jiongmin Yong. A linear-quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon. Mathematical Control & Related Fields, 2015, 5 (1) : 97-139. doi: 10.3934/mcrf.2015.5.97
[5]

Jun Moon. Linear-quadratic mean-field type stackelberg differential games for stochastic jump-diffusion systems. Mathematical Control & Related Fields, 2021  doi: 10.3934/mcrf.2021026
[6]

Nguyen Thi Hoai. Asymptotic approximation to a solution of a singularly perturbed linear-quadratic optimal control problem with second-order linear ordinary differential equation of state variable. Numerical Algebra, Control & Optimization, 2021, 11 (4) : 495-512. doi: 10.3934/naco.2020040
[7]

Kai Du, Jianhui Huang, Zhen Wu. Linear quadratic mean-field-game of backward stochastic differential systems. Mathematical Control & Related Fields, 2018, 8 (3&4) : 653-678. doi: 10.3934/mcrf.2018028
[8]

Tianxiao Wang. Characterizations of equilibrium controls in time inconsistent mean-field stochastic linear quadratic problems. I. Mathematical Control & Related Fields, 2019, 9 (2) : 385-409. doi: 10.3934/mcrf.2019018
[9]

Zhenghong Qiu, Jianhui Huang, Tinghan Xie. Linear-Quadratic-Gaussian mean-field controls of social optima. Mathematical Control & Related Fields, 2021  doi: 10.3934/mcrf.2021047
[10]

Alain Bensoussan, Shaokuan Chen, Suresh P. Sethi. Linear quadratic differential games with mixed leadership: The open-loop solution. Numerical Algebra, Control & Optimization, 2013, 3 (1) : 95-108. doi: 10.3934/naco.2013.3.95
[11]

Michael Herty, Lorenzo Pareschi, Sonja Steffensen. Mean--field control and Riccati equations. Networks & Heterogeneous Media, 2015, 10 (3) : 699-715. doi: 10.3934/nhm.2015.10.699
[12]

Diana Keller. Optimal control of a linear stochastic Schrödinger equation. Conference Publications, 2013, 2013 (special) : 437-446. doi: 10.3934/proc.2013.2013.437
[13]

Jianhui Huang, Shujun Wang, Zhen Wu. Backward-forward linear-quadratic mean-field games with major and minor agents. Probability, Uncertainty and Quantitative Risk, 2016, 1 (0) : 8-. doi: 10.1186/s41546-016-0009-9
[14]

René Carmona, Kenza Hamidouche, Mathieu Laurière, Zongjun Tan. Linear-quadratic zero-sum mean-field type games: Optimality conditions and policy optimization. Journal of Dynamics & Games, 2021, 8 (4) : 403-443. doi: 10.3934/jdg.2021023
[15]

Haiyan Zhang. A necessary condition for mean-field type stochastic differential equations with correlated state and observation noises. Journal of Industrial & Management Optimization, 2016, 12 (4) : 1287-1301. doi: 10.3934/jimo.2016.12.1287
[16]

Juan Li, Wenqiang Li. Controlled reflected mean-field backward stochastic differential equations coupled with value function and related PDEs. Mathematical Control & Related Fields, 2015, 5 (3) : 501-516. doi: 10.3934/mcrf.2015.5.501
[17]

Hancheng Guo, Jie Xiong. A second-order stochastic maximum principle for generalized mean-field singular control problem. Mathematical Control & Related Fields, 2018, 8 (2) : 451-473. doi: 10.3934/mcrf.2018018
[18]

Salah Eddine Choutri, Boualem Djehiche, Hamidou Tembine. Optimal control and zero-sum games for Markov chains of mean-field type. Mathematical Control & Related Fields, 2019, 9 (3) : 571-605. doi: 10.3934/mcrf.2019026
[19]

Simon Hochgerner. Symmetry actuated closed-loop Hamiltonian systems. Journal of Geometric Mechanics, 2020, 12 (4) : 641-669. doi: 10.3934/jgm.2020030
[20]

Pasquale Palumbo, Pierdomenico Pepe, Simona Panunzi, Andrea De Gaetano. Robust closed-loop control of plasma glycemia: A discrete-delay model approach. Discrete & Continuous Dynamical Systems - B, 2009, 12 (2) : 455-468. doi: 10.3934/dcdsb.2009.12.455

 Impact Factor: 

Tools

Metrics

  • PDF downloads (22)
  • HTML views (0)
  • Cited by (13)

Other articles
by authors

[Back to Top]

Copyright © 2021 American Institute of Mathematical Sciences | Privacy Policy