American Institute of Mathematical Sciences

Advanced
January  2016, 1: 2 doi: 10.1186/s41546-016-0002-3

Mean-field stochastic linear quadratic optimal control problems: closed-loop solvability

Xun Li 1, , Jingrui Sun 1, and  Jiongmin Yong 2,,

1 Department of Applied Mathematics, The Hong Kong Polytechnic University, Hong Kong, China;

2 Department of Mathematics, University of Central Florida, Orlando, FL 32816, USA

Received  February 24, 2016 Revised  June 10, 2016 Published  August 2016

Fund Project: supported by Hong Kong RGC under grants 519913, 15209614 and 15224215. Jingrui Sun was partially supported by the National Natural Science Foundation of China (11401556) and the Fundamental Research Funds for the Central Universities (WK 2040000012). Jiongmin Yong was partially supported by NSF DMS-1406776.

An optimal control problem is studied for a linear mean-field stochastic differential equation with a quadratic cost functional. The coefficients and the weighting matrices in the cost functional are all assumed to be deterministic. Closedloop strategies are introduced, which require to be independent of initial states; and such a nature makes it very useful and convenient in applications. In this paper, the existence of an optimal closed-loop strategy for the system (also called the closedloop solvability of the problem) is characterized by the existence of a regular solution to the coupled two (generalized) Riccati equations, together with some constraints on the adapted solution to a linear backward stochastic differential equation and a linear terminal value problem of an ordinary differential equation.
Keywords: Mean-field stochastic differential equation, Linear quadratic optimal control, Riccati equation, Regular solution, Closed-loop solvability.
Mathematics Subject Classification: 49N10;49N35;93E20.
Citation: Xun Li, Jingrui Sun, Jiongmin Yong. Mean-field stochastic linear quadratic optimal control problems: closed-loop solvability. Probability, Uncertainty and Quantitative Risk, 2016, 1 (0) : 2-. doi: 10.1186/s41546-016-0002-3
References:
[1]

Ait Rami, M, Moore, JB, Zhou, XY:Indefinite stochastic linear quadratic control and generalized differential Riccati equation. SIAM J. Control Optim 40, 1296-1311 (2001)

[2]

Andersson, D, Djehiche, B:A maximum principle for SDEs of mean-field type. Appl. Math. Optim 63, 341-356 (2011)

[3]

Athans, M:The matrix minimum principle. Inform. Control 11, 592-606 (1968)

[4]

Buckdahn, R, Djehiche, B, Li, J:A general stochastic maximum principle for SDEs of mean-field type.Appl. Math. Optim 64, 197-216 (2011)

[5]

Buckdahn, R, Djehiche, B, Li, J, Peng, S:Mean-field backward stochastic differential equations:a limit approach. Ann. Probab 37, 1524-1565 (2009)

[6]

Buckdahn, R, Li, J, Peng, S:Mean-field backward stochastic differential equations and related partial differential equations. Stoch. Proc. Appl 119, 3133-3154 (2009)

[7]

Chen, S, Li, X, Zhou, XY:Stochastic linear quadratic regulators with indefinite control weight costs.SIAM J. Control Optim 36, 1685-1702 (1998)

[8]

Chen, S, Yong, J:Stochastic linear quadratic optimal control problems with random coefficients. Chin.Ann. Math 21B, 323-338 (2000)

[9]

Cui, XY, Li, X, Li, D:Unified framework of mean-field formulations for optimal multi-period meanvariance portfolio selection. IEEE Trans. Auto. Control 59, 1833-1844 (2014)

[10]

Elliott, R, Li, X, Ni, YH:Discrete time mean-field stochastic linear-quadratic optimal control problems.Automatica 49, 3222-3233 (2013)

[11]

Huang, J, Li, X, Wang, TX:Mean-field linear-quadratic-Gaussian (LQG) games for stochastic integral systems. IEEE Trans. Auto. Control (2015). doi:10.1109/TAC.2015.2506620

[12]

Huang, J, Li, X, Yong, J:A linear-quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon. Math. Control Related Fields 5, 97-139 (2015)

[13]

Kac, M:Foundations of kinetic theory. Proc. Third Berkeley Symp. Math. Stat. Probab 3, 171-197 (1956)

[14]

McKean, HP:A class of Markov processes associated with nonlinear parabolic equations. Proc. Natl.Acad. Sci. USA 56, 1907-1911 (1966)

[15]

Meyer-Brandis, T, Øksendal, B, Zhou, XY:A mean-field stochastic maximum principle via Malliavin calculus. Stochastics 84(5-6), 643-666 (2012). doi:10.1080/17442508.2011.651619

[16]

Penrose, R:A generalized inverse of matrices. Proc. Cambridge Philos Soc 52, 17-19 (1955)

[17]

Sun, J:Mean-field stochastic linear quadratic optimal control problems:open-loop solvabilities. ESAIM:COCV, 016023 (2016). doi:10.1051/cocv/2

[18]

Sun, J, Yong, J:Linear quadratic stochastic differential games:open-loop and closed-loop saddle points.SIAM J. Control Optim 52, 4082-4121 (2014)

[19]

Sun, J, Yong, J, Zhang, S:Linear quadratic stochastic two-person zero-sum differential games in an infinite horizon. ESAIM COCV 22, 743-769 (2016). doi:10.1051/cocv/2015024

[20]

Tang, S:General linear quadratic optimal stochastic control problems with random coefficients:linear stochastic Hamilton systems and backward stochastic Riccati equations. SIAM J. Control Optim 42, 53-75 (2003)

[21]

Wonham, WM:On a matrix Riccati equation of stochastic control. SIAM J. Control Optim 6, 681-697(1968)

[22]

Yong, J:Linear-quadratic optimal control problems for mean-field stochastic differential equations. SIAM J. Control Optim 51, 2809-2838 (2013)

[23]

Yong, J, Zhou, XY:Stochastic controls:Hamiltonian systems and HJB equations. Springer-Verlag, New York (1999)

show all references

References:
[1]

Ait Rami, M, Moore, JB, Zhou, XY:Indefinite stochastic linear quadratic control and generalized differential Riccati equation. SIAM J. Control Optim 40, 1296-1311 (2001)

[2]

Andersson, D, Djehiche, B:A maximum principle for SDEs of mean-field type. Appl. Math. Optim 63, 341-356 (2011)

[3]

Athans, M:The matrix minimum principle. Inform. Control 11, 592-606 (1968)

[4]

Buckdahn, R, Djehiche, B, Li, J:A general stochastic maximum principle for SDEs of mean-field type.Appl. Math. Optim 64, 197-216 (2011)

[5]

Buckdahn, R, Djehiche, B, Li, J, Peng, S:Mean-field backward stochastic differential equations:a limit approach. Ann. Probab 37, 1524-1565 (2009)

[6]

Buckdahn, R, Li, J, Peng, S:Mean-field backward stochastic differential equations and related partial differential equations. Stoch. Proc. Appl 119, 3133-3154 (2009)

[7]

Chen, S, Li, X, Zhou, XY:Stochastic linear quadratic regulators with indefinite control weight costs.SIAM J. Control Optim 36, 1685-1702 (1998)

[8]

Chen, S, Yong, J:Stochastic linear quadratic optimal control problems with random coefficients. Chin.Ann. Math 21B, 323-338 (2000)

[9]

Cui, XY, Li, X, Li, D:Unified framework of mean-field formulations for optimal multi-period meanvariance portfolio selection. IEEE Trans. Auto. Control 59, 1833-1844 (2014)

[10]

Elliott, R, Li, X, Ni, YH:Discrete time mean-field stochastic linear-quadratic optimal control problems.Automatica 49, 3222-3233 (2013)

[11]

Huang, J, Li, X, Wang, TX:Mean-field linear-quadratic-Gaussian (LQG) games for stochastic integral systems. IEEE Trans. Auto. Control (2015). doi:10.1109/TAC.2015.2506620

[12]

Huang, J, Li, X, Yong, J:A linear-quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon. Math. Control Related Fields 5, 97-139 (2015)

[13]

Kac, M:Foundations of kinetic theory. Proc. Third Berkeley Symp. Math. Stat. Probab 3, 171-197 (1956)

[14]

McKean, HP:A class of Markov processes associated with nonlinear parabolic equations. Proc. Natl.Acad. Sci. USA 56, 1907-1911 (1966)

[15]

Meyer-Brandis, T, Øksendal, B, Zhou, XY:A mean-field stochastic maximum principle via Malliavin calculus. Stochastics 84(5-6), 643-666 (2012). doi:10.1080/17442508.2011.651619

[16]

Penrose, R:A generalized inverse of matrices. Proc. Cambridge Philos Soc 52, 17-19 (1955)

[17]

Sun, J:Mean-field stochastic linear quadratic optimal control problems:open-loop solvabilities. ESAIM:COCV, 016023 (2016). doi:10.1051/cocv/2

[18]

Sun, J, Yong, J:Linear quadratic stochastic differential games:open-loop and closed-loop saddle points.SIAM J. Control Optim 52, 4082-4121 (2014)

[19]

Sun, J, Yong, J, Zhang, S:Linear quadratic stochastic two-person zero-sum differential games in an infinite horizon. ESAIM COCV 22, 743-769 (2016). doi:10.1051/cocv/2015024

[20]

Tang, S:General linear quadratic optimal stochastic control problems with random coefficients:linear stochastic Hamilton systems and backward stochastic Riccati equations. SIAM J. Control Optim 42, 53-75 (2003)

[21]

Wonham, WM:On a matrix Riccati equation of stochastic control. SIAM J. Control Optim 6, 681-697(1968)

[22]

Yong, J:Linear-quadratic optimal control problems for mean-field stochastic differential equations. SIAM J. Control Optim 51, 2809-2838 (2013)

[23]

Yong, J, Zhou, XY:Stochastic controls:Hamiltonian systems and HJB equations. Springer-Verlag, New York (1999)

[1]

Jingrui Sun, Hanxiao Wang. Mean-field stochastic linear-quadratic optimal control problems: Weak closed-loop solvability. Mathematical Control and Related Fields, 2021, 11 (1) : 47-71. doi: 10.3934/mcrf.2020026
[2]

Hanxiao Wang, Jingrui Sun, Jiongmin Yong. Weak closed-loop solvability of stochastic linear-quadratic optimal control problems. Discrete and Continuous Dynamical Systems, 2019, 39 (5) : 2785-2805. doi: 10.3934/dcds.2019117
[3]

Kehan Si, Zhenda Xu, Ka Fai Cedric Yiu, Xun Li. Open-loop solvability for mean-field stochastic linear quadratic optimal control problems of Markov regime-switching system. Journal of Industrial and Management Optimization, 2021  doi: 10.3934/jimo.2021074
[4]

Jianhui Huang, Xun Li, Jiongmin Yong. A linear-quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon. Mathematical Control and Related Fields, 2015, 5 (1) : 97-139. doi: 10.3934/mcrf.2015.5.97
[5]

Jun Moon. Linear-quadratic mean-field type stackelberg differential games for stochastic jump-diffusion systems. Mathematical Control and Related Fields, 2022, 12 (2) : 371-404. doi: 10.3934/mcrf.2021026
[6]

Nguyen Thi Hoai. Asymptotic approximation to a solution of a singularly perturbed linear-quadratic optimal control problem with second-order linear ordinary differential equation of state variable. Numerical Algebra, Control and Optimization, 2021, 11 (4) : 495-512. doi: 10.3934/naco.2020040
[7]

Kai Du, Jianhui Huang, Zhen Wu. Linear quadratic mean-field-game of backward stochastic differential systems. Mathematical Control and Related Fields, 2018, 8 (3&4) : 653-678. doi: 10.3934/mcrf.2018028
[8]

Tianxiao Wang. Characterizations of equilibrium controls in time inconsistent mean-field stochastic linear quadratic problems. I. Mathematical Control and Related Fields, 2019, 9 (2) : 385-409. doi: 10.3934/mcrf.2019018
[9]

Zhenghong Qiu, Jianhui Huang, Tinghan Xie. Linear-Quadratic-Gaussian mean-field controls of social optima. Mathematical Control and Related Fields, 2021  doi: 10.3934/mcrf.2021047
[10]

Alain Bensoussan, Shaokuan Chen, Suresh P. Sethi. Linear quadratic differential games with mixed leadership: The open-loop solution. Numerical Algebra, Control and Optimization, 2013, 3 (1) : 95-108. doi: 10.3934/naco.2013.3.95
[11]

Michael Herty, Lorenzo Pareschi, Sonja Steffensen. Mean--field control and Riccati equations. Networks and Heterogeneous Media, 2015, 10 (3) : 699-715. doi: 10.3934/nhm.2015.10.699
[12]

Diana Keller. Optimal control of a linear stochastic Schrödinger equation. Conference Publications, 2013, 2013 (special) : 437-446. doi: 10.3934/proc.2013.2013.437
[13]

Hélène Hibon, Ying Hu, Shanjian Tang. Mean-field type quadratic BSDEs. Numerical Algebra, Control and Optimization, 2022  doi: 10.3934/naco.2022009
[14]

Jianhui Huang, Shujun Wang, Zhen Wu. Backward-forward linear-quadratic mean-field games with major and minor agents. Probability, Uncertainty and Quantitative Risk, 2016, 1 (0) : 8-. doi: 10.1186/s41546-016-0009-9
[15]

René Carmona, Kenza Hamidouche, Mathieu Laurière, Zongjun Tan. Linear-quadratic zero-sum mean-field type games: Optimality conditions and policy optimization. Journal of Dynamics and Games, 2021, 8 (4) : 403-443. doi: 10.3934/jdg.2021023
[16]

Haiyan Zhang. A necessary condition for mean-field type stochastic differential equations with correlated state and observation noises. Journal of Industrial and Management Optimization, 2016, 12 (4) : 1287-1301. doi: 10.3934/jimo.2016.12.1287
[17]

Juan Li, Wenqiang Li. Controlled reflected mean-field backward stochastic differential equations coupled with value function and related PDEs. Mathematical Control and Related Fields, 2015, 5 (3) : 501-516. doi: 10.3934/mcrf.2015.5.501
[18]

Hancheng Guo, Jie Xiong. A second-order stochastic maximum principle for generalized mean-field singular control problem. Mathematical Control and Related Fields, 2018, 8 (2) : 451-473. doi: 10.3934/mcrf.2018018
[19]

Salah Eddine Choutri, Boualem Djehiche, Hamidou Tembine. Optimal control and zero-sum games for Markov chains of mean-field type. Mathematical Control and Related Fields, 2019, 9 (3) : 571-605. doi: 10.3934/mcrf.2019026
[20]

Simon Hochgerner. Symmetry actuated closed-loop Hamiltonian systems. Journal of Geometric Mechanics, 2020, 12 (4) : 641-669. doi: 10.3934/jgm.2020030

 Impact Factor: 

Tools

Metrics

  • PDF downloads (34)
  • HTML views (0)
  • Cited by (13)

Other articles
by authors

[Back to Top]

Copyright © 2022 American Institute of Mathematical Sciences | Privacy Policy