March  2015, 5(1): 97-139. doi: 10.3934/mcrf.2015.5.97

A linear-quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon

1. 

Department of Applied Mathematics, The Hong Kong Polytechnic University, Hong Kong, China, China

2. 

Department of Mathematics, University of Central Florida, Orlando, FL 32816

Received  July 2013 Revised  November 2013 Published  January 2015

A linear-quadratic (LQ, for short) optimal control problem is considered for mean-field stochastic differential equations with constant coefficients in an infinite horizon. The stabilizability of the control system is studied followed by the discussion of the well-posedness of the LQ problem. The optimal control can be expressed as a linear state feedback involving the state and its mean, through the solutions of two algebraic Riccati equations. The solvability of such kind of Riccati equations is investigated by means of semi-definite programming method.
Citation: Jianhui Huang, Xun Li, Jiongmin Yong. A linear-quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon. Mathematical Control and Related Fields, 2015, 5 (1) : 97-139. doi: 10.3934/mcrf.2015.5.97
References:
[1]

N. U. Ahmed and X. Ding, A semilinear McKean-Vlasov stochastic evolution equation in Hilbert space, Stoch. Proc. Appl., 60 (1995), 65-85. doi: 10.1016/0304-4149(95)00050-X.

[2]

N. U. Ahmed and X. Ding, Controlled McKean-Vlasov equations, Comm. Appl. Anal., 5 (2001), 183-206.

[3]

N. U. Ahmed, Nonlinear diffusion governed by McKean-Vlasov equation on Hilbert space and optimal control, SIAM J. Control Optim., 46 (2007), 356-378. doi: 10.1137/050645944.

[4]

M. Ait Rami and X. Y. Zhou, Linear matrix inequalities, Riccati equations, and indefinite stochastic linear quadratic controls, IEEE Transactions on Automatic Control, 45 (2000), 1131-1143. doi: 10.1109/9.863597.

[5]

A. Albert, Conditions for positive and nonnegative definiteness in terms of pseudo-inverses, SIAM J. Appl. Math., 17 (1969), 434-440. doi: 10.1137/0117041.

[6]

D. Andersson and B. Djehiche, A maximum principle for SDEs of mean-field type, Appl. Math. Optim., 63 (2011), 341-356. doi: 10.1007/s00245-010-9123-8.

[7]

A. Ben-Israel and T. N. E. Greville, Generalized Inverses, Springer-Verlag, 2003.

[8]

A. Bensoussan, G. Da Prato, M. C. Delfour and S. K. Mitter, Representation and Control of Infinite Dimensional Systems, 2nd edition, Systems & Control: Foundations & Applications. Birkhäuser Boston, Inc., Boston, 2007. doi: 10.1007/978-0-8176-4581-6.

[9]

V. S. Borkar and K. S. Kumar, McKean-Vlasov limit in portfolio optimization, Stoch. Anal. Appl., 28 (2010), 884-906. doi: 10.1080/07362994.2010.482836.

[10]

S. Boyd, L. El Ghaoul, E. Feron and V. Balakrishnan, Linear Matrix Inequality in Systems and Control Theory, SIAM, Philadelphia, 1994. doi: 10.1137/1.9781611970777.

[11]

R. Buckdahn, B. Djehiche and J. Li, A general maximum principle for SDEs of mean-field type, Applied Mathematics & Optimization, 64 (2011), 197-216. doi: 10.1007/s00245-011-9136-y.

[12]

R. Buckdahn, B. Djehiche, J. Li and S. Peng, Mean-field backward stochastic differential equations: A limit approach, Ann. Probab., 37 (2009), 1524-1565. doi: 10.1214/08-AOP442.

[13]

R. Buckdahn, J. Li and S. Peng, Mean-field backward stochastic differential equations and related partial differential equations, Stoch. Process. Appl., 119 (2009), 3133-3154. doi: 10.1016/j.spa.2009.05.002.

[14]

T. Chan, Dynamics of the McKean-Vlasov equation, Ann. Probab., 22 (1994), 431-441. doi: 10.1214/aop/1176988866.

[15]

T. Chiang, McKean-Vlasov equations with discontinuous coefficients, Soochow J. Math., 20 (1994), 507-526.

[16]

D. Crisan and J. Xiong, Approximate McKean-Vlasov representations for a class of SPDEs, Stochastics, 82 (2010), 53-68. doi: 10.1080/17442500902723575.

[17]

D. A. Dawson, Critical dynamics and fluctuations for a mean-field model of cooperative behavior, J. Statist. Phys., 31 (1983), 29-85. doi: 10.1007/BF01010922.

[18]

D. A. Dawson and J. Gärtner, Large deviations from the McKean-Vlasov limit for weakly interacting diffusions, Stochastics, 20 (1987), 247-308. doi: 10.1080/17442508708833446.

[19]

L. El Ghaoui and M. Ait Rami, Robust state-feedback stabilization of jump linear systems via LIMs, Int. J. Robust and Nonlinear Contr., 6 (1996), 1015-1022. doi: 10.1002/(SICI)1099-1239(199611)6:9/10<1015::AID-RNC266>3.0.CO;2-0.

[20]

J. Gärtner, On the Mckean-Vlasov limit for interacting diffusions, Math. Nachr., 137 (1988), 197-248. doi: 10.1002/mana.19881370116.

[21]

C. Graham, McKean-Vlasov Ito-Skorohod equations, and nonlinear diffusions with discrete jump sets, Stoch. Proc. Appl., 40 (1992), 69-82. doi: 10.1016/0304-4149(92)90138-G.

[22]

M. Grant and S. Boyd, Graph implementations for nonsmooth convex programs, Recent Advances in Learning and Control (a tribute to M. Vidyasagar), Lecture Notes in Control and Information Sciences, Springer, 371 (2008), 95-110. doi: 10.1007/978-1-84800-155-8_7.

[23]

M. Huang, R. P. Malhamé, and P. E. Caines, Large population stochastic dynamic games: Closed-loop McKean-Vlasov systems and the Nash certainty equivalence principle, Comm. Inform. Systems, 6 (2006), 221-251. doi: 10.4310/CIS.2006.v6.n3.a5.

[24]

M. Kac, Foundations of kinetic theory, Proc. 3rd Berkeley Sympos. Math. Statist. Prob., 3 (1956), 171-197.

[25]

P. E. Kloeden and T. Lorenz, Stochastic differential equations with nonlocal sample dependence, Stoch. Anal. Appl., 28 (2010), 937-945. doi: 10.1080/07362994.2010.515194.

[26]

P. M. Kotelenez and T. G. Kurtz, Macroscopic limit for stochastic partial differential equations of McKean-Vlasov type, Prob. Theory Rel. Fields, 146 (2010), 189-222. doi: 10.1007/s00440-008-0188-0.

[27]

J. M. Lasry and P. L. Lions, Mean field games, Japan J. Math., 2 (2007), 229-260. doi: 10.1007/s11537-007-0657-8.

[28]

X. Li, X.Y. Zhou and M. Ait Rami, Indefinite stochastic linear quadratic control with Markovian jumps in infinite time horizon, Journal of Global Optimization, 27 (2003), 149-175. doi: 10.1023/A:1024887007165.

[29]

H. P. McKean, A class of Markov processes associated with nonlinear parabolic equations, Proc. Natl. Acad. Sci. USA, 56 (1966), 1907-1911. doi: 10.1073/pnas.56.6.1907.

[30]

T. Meyer-Brandis, B. /'Oksendal and X. Y. Zhou, A mean-field stochastic maximum principle via Malliavin calculus, Stochastics, 84 (2012), 643-666. doi: 10.1080/17442508.2011.651619.

[31]

J. Y. Park, P. Balasubramaniam and Y. H. Kang, Controllability of McKean-Vlasov stochastic integrodifferential evolution equation in Hilbert spaces, Numer. Funct. Anal. Optim., 29 (2008), 1328-1346. doi: 10.1080/01630560802580679.

[32]

R. Penrose, A generalized inverse of matrices, Proc. Cambridge Philos. Soc., 51 (1955), 406-413. doi: 10.1017/S0305004100030401.

[33]

M. Scheutzow, Uniqueness and non-uniqueness of solutions of Vlasov-McKean equations, J. Austral. Math. Soc., Ser. A, 43 (1987), 246-256. doi: 10.1017/S1446788700029384.

[34]

R. H Tutuncu, K. C. Toh, and M. J. Todd, Solving semidefinite-quadratic-linear programs using SDPT3, Mathematical Programming Ser. B, 95 (2003), 189-217. doi: 10.1007/s10107-002-0347-5.

[35]

L. Vandenerghe and S. Boyd, Semidefinite programming, SIAM Rev., 38 (1996), 49-95. doi: 10.1137/1038003.

[36]

A. Yu. Veretennikov, On ergodic measures for McKean-Vlasov stochastic equations, in Monte Carlo and quasi-Monte Carlo methods 2004, Springer, Berlin, 2006, 471-486. doi: 10.1007/3-540-31186-6_29.

[37]

J. Yong, Linear-quadratic optimal control problems for mean-field stochastic differential equations, SIAM J. Control Optim., 51 (2013), 2809-2838. doi: 10.1137/120892477.

[38]

J. Yong and X. Y. Zhou, Stochastic Controls: Hamiltonian Systems and HJB Equations, Applications of Mathematics (New York), 43, Springer-Verlag, New York, 1999. doi: 10.1007/978-1-4612-1466-3.

show all references

References:
[1]

N. U. Ahmed and X. Ding, A semilinear McKean-Vlasov stochastic evolution equation in Hilbert space, Stoch. Proc. Appl., 60 (1995), 65-85. doi: 10.1016/0304-4149(95)00050-X.

[2]

N. U. Ahmed and X. Ding, Controlled McKean-Vlasov equations, Comm. Appl. Anal., 5 (2001), 183-206.

[3]

N. U. Ahmed, Nonlinear diffusion governed by McKean-Vlasov equation on Hilbert space and optimal control, SIAM J. Control Optim., 46 (2007), 356-378. doi: 10.1137/050645944.

[4]

M. Ait Rami and X. Y. Zhou, Linear matrix inequalities, Riccati equations, and indefinite stochastic linear quadratic controls, IEEE Transactions on Automatic Control, 45 (2000), 1131-1143. doi: 10.1109/9.863597.

[5]

A. Albert, Conditions for positive and nonnegative definiteness in terms of pseudo-inverses, SIAM J. Appl. Math., 17 (1969), 434-440. doi: 10.1137/0117041.

[6]

D. Andersson and B. Djehiche, A maximum principle for SDEs of mean-field type, Appl. Math. Optim., 63 (2011), 341-356. doi: 10.1007/s00245-010-9123-8.

[7]

A. Ben-Israel and T. N. E. Greville, Generalized Inverses, Springer-Verlag, 2003.

[8]

A. Bensoussan, G. Da Prato, M. C. Delfour and S. K. Mitter, Representation and Control of Infinite Dimensional Systems, 2nd edition, Systems & Control: Foundations & Applications. Birkhäuser Boston, Inc., Boston, 2007. doi: 10.1007/978-0-8176-4581-6.

[9]

V. S. Borkar and K. S. Kumar, McKean-Vlasov limit in portfolio optimization, Stoch. Anal. Appl., 28 (2010), 884-906. doi: 10.1080/07362994.2010.482836.

[10]

S. Boyd, L. El Ghaoul, E. Feron and V. Balakrishnan, Linear Matrix Inequality in Systems and Control Theory, SIAM, Philadelphia, 1994. doi: 10.1137/1.9781611970777.

[11]

R. Buckdahn, B. Djehiche and J. Li, A general maximum principle for SDEs of mean-field type, Applied Mathematics & Optimization, 64 (2011), 197-216. doi: 10.1007/s00245-011-9136-y.

[12]

R. Buckdahn, B. Djehiche, J. Li and S. Peng, Mean-field backward stochastic differential equations: A limit approach, Ann. Probab., 37 (2009), 1524-1565. doi: 10.1214/08-AOP442.

[13]

R. Buckdahn, J. Li and S. Peng, Mean-field backward stochastic differential equations and related partial differential equations, Stoch. Process. Appl., 119 (2009), 3133-3154. doi: 10.1016/j.spa.2009.05.002.

[14]

T. Chan, Dynamics of the McKean-Vlasov equation, Ann. Probab., 22 (1994), 431-441. doi: 10.1214/aop/1176988866.

[15]

T. Chiang, McKean-Vlasov equations with discontinuous coefficients, Soochow J. Math., 20 (1994), 507-526.

[16]

D. Crisan and J. Xiong, Approximate McKean-Vlasov representations for a class of SPDEs, Stochastics, 82 (2010), 53-68. doi: 10.1080/17442500902723575.

[17]

D. A. Dawson, Critical dynamics and fluctuations for a mean-field model of cooperative behavior, J. Statist. Phys., 31 (1983), 29-85. doi: 10.1007/BF01010922.

[18]

D. A. Dawson and J. Gärtner, Large deviations from the McKean-Vlasov limit for weakly interacting diffusions, Stochastics, 20 (1987), 247-308. doi: 10.1080/17442508708833446.

[19]

L. El Ghaoui and M. Ait Rami, Robust state-feedback stabilization of jump linear systems via LIMs, Int. J. Robust and Nonlinear Contr., 6 (1996), 1015-1022. doi: 10.1002/(SICI)1099-1239(199611)6:9/10<1015::AID-RNC266>3.0.CO;2-0.

[20]

J. Gärtner, On the Mckean-Vlasov limit for interacting diffusions, Math. Nachr., 137 (1988), 197-248. doi: 10.1002/mana.19881370116.

[21]

C. Graham, McKean-Vlasov Ito-Skorohod equations, and nonlinear diffusions with discrete jump sets, Stoch. Proc. Appl., 40 (1992), 69-82. doi: 10.1016/0304-4149(92)90138-G.

[22]

M. Grant and S. Boyd, Graph implementations for nonsmooth convex programs, Recent Advances in Learning and Control (a tribute to M. Vidyasagar), Lecture Notes in Control and Information Sciences, Springer, 371 (2008), 95-110. doi: 10.1007/978-1-84800-155-8_7.

[23]

M. Huang, R. P. Malhamé, and P. E. Caines, Large population stochastic dynamic games: Closed-loop McKean-Vlasov systems and the Nash certainty equivalence principle, Comm. Inform. Systems, 6 (2006), 221-251. doi: 10.4310/CIS.2006.v6.n3.a5.

[24]

M. Kac, Foundations of kinetic theory, Proc. 3rd Berkeley Sympos. Math. Statist. Prob., 3 (1956), 171-197.

[25]

P. E. Kloeden and T. Lorenz, Stochastic differential equations with nonlocal sample dependence, Stoch. Anal. Appl., 28 (2010), 937-945. doi: 10.1080/07362994.2010.515194.

[26]

P. M. Kotelenez and T. G. Kurtz, Macroscopic limit for stochastic partial differential equations of McKean-Vlasov type, Prob. Theory Rel. Fields, 146 (2010), 189-222. doi: 10.1007/s00440-008-0188-0.

[27]

J. M. Lasry and P. L. Lions, Mean field games, Japan J. Math., 2 (2007), 229-260. doi: 10.1007/s11537-007-0657-8.

[28]

X. Li, X.Y. Zhou and M. Ait Rami, Indefinite stochastic linear quadratic control with Markovian jumps in infinite time horizon, Journal of Global Optimization, 27 (2003), 149-175. doi: 10.1023/A:1024887007165.

[29]

H. P. McKean, A class of Markov processes associated with nonlinear parabolic equations, Proc. Natl. Acad. Sci. USA, 56 (1966), 1907-1911. doi: 10.1073/pnas.56.6.1907.

[30]

T. Meyer-Brandis, B. /'Oksendal and X. Y. Zhou, A mean-field stochastic maximum principle via Malliavin calculus, Stochastics, 84 (2012), 643-666. doi: 10.1080/17442508.2011.651619.

[31]

J. Y. Park, P. Balasubramaniam and Y. H. Kang, Controllability of McKean-Vlasov stochastic integrodifferential evolution equation in Hilbert spaces, Numer. Funct. Anal. Optim., 29 (2008), 1328-1346. doi: 10.1080/01630560802580679.

[32]

R. Penrose, A generalized inverse of matrices, Proc. Cambridge Philos. Soc., 51 (1955), 406-413. doi: 10.1017/S0305004100030401.

[33]

M. Scheutzow, Uniqueness and non-uniqueness of solutions of Vlasov-McKean equations, J. Austral. Math. Soc., Ser. A, 43 (1987), 246-256. doi: 10.1017/S1446788700029384.

[34]

R. H Tutuncu, K. C. Toh, and M. J. Todd, Solving semidefinite-quadratic-linear programs using SDPT3, Mathematical Programming Ser. B, 95 (2003), 189-217. doi: 10.1007/s10107-002-0347-5.

[35]

L. Vandenerghe and S. Boyd, Semidefinite programming, SIAM Rev., 38 (1996), 49-95. doi: 10.1137/1038003.

[36]

A. Yu. Veretennikov, On ergodic measures for McKean-Vlasov stochastic equations, in Monte Carlo and quasi-Monte Carlo methods 2004, Springer, Berlin, 2006, 471-486. doi: 10.1007/3-540-31186-6_29.

[37]

J. Yong, Linear-quadratic optimal control problems for mean-field stochastic differential equations, SIAM J. Control Optim., 51 (2013), 2809-2838. doi: 10.1137/120892477.

[38]

J. Yong and X. Y. Zhou, Stochastic Controls: Hamiltonian Systems and HJB Equations, Applications of Mathematics (New York), 43, Springer-Verlag, New York, 1999. doi: 10.1007/978-1-4612-1466-3.

[1]

Jingrui Sun, Hanxiao Wang. Mean-field stochastic linear-quadratic optimal control problems: Weak closed-loop solvability. Mathematical Control and Related Fields, 2021, 11 (1) : 47-71. doi: 10.3934/mcrf.2020026

[2]

Jun Moon. Linear-quadratic mean-field type stackelberg differential games for stochastic jump-diffusion systems. Mathematical Control and Related Fields, 2022, 12 (2) : 371-404. doi: 10.3934/mcrf.2021026

[3]

Xun Li, Jingrui Sun, Jiongmin Yong. Mean-field stochastic linear quadratic optimal control problems: closed-loop solvability. Probability, Uncertainty and Quantitative Risk, 2016, 1 (0) : 2-. doi: 10.1186/s41546-016-0002-3

[4]

Kehan Si, Zhenda Xu, Ka Fai Cedric Yiu, Xun Li. Open-loop solvability for mean-field stochastic linear quadratic optimal control problems of Markov regime-switching system. Journal of Industrial and Management Optimization, 2022, 18 (4) : 2415-2433. doi: 10.3934/jimo.2021074

[5]

Nguyen Thi Hoai. Asymptotic approximation to a solution of a singularly perturbed linear-quadratic optimal control problem with second-order linear ordinary differential equation of state variable. Numerical Algebra, Control and Optimization, 2021, 11 (4) : 495-512. doi: 10.3934/naco.2020040

[6]

Jianhui Huang, Shujun Wang, Zhen Wu. Backward-forward linear-quadratic mean-field games with major and minor agents. Probability, Uncertainty and Quantitative Risk, 2016, 1 (0) : 8-. doi: 10.1186/s41546-016-0009-9

[7]

René Carmona, Kenza Hamidouche, Mathieu Laurière, Zongjun Tan. Linear-quadratic zero-sum mean-field type games: Optimality conditions and policy optimization. Journal of Dynamics and Games, 2021, 8 (4) : 403-443. doi: 10.3934/jdg.2021023

[8]

Kai Du, Jianhui Huang, Zhen Wu. Linear quadratic mean-field-game of backward stochastic differential systems. Mathematical Control and Related Fields, 2018, 8 (3&4) : 653-678. doi: 10.3934/mcrf.2018028

[9]

Tianxiao Wang. Characterizations of equilibrium controls in time inconsistent mean-field stochastic linear quadratic problems. I. Mathematical Control and Related Fields, 2019, 9 (2) : 385-409. doi: 10.3934/mcrf.2019018

[10]

Martino Bardi. Explicit solutions of some linear-quadratic mean field games. Networks and Heterogeneous Media, 2012, 7 (2) : 243-261. doi: 10.3934/nhm.2012.7.243

[11]

Zhenghong Qiu, Jianhui Huang, Tinghan Xie. Linear-Quadratic-Gaussian mean-field controls of social optima. Mathematical Control and Related Fields, 2021  doi: 10.3934/mcrf.2021047

[12]

Thierry Horsin, Peter I. Kogut, Olivier Wilk. Optimal $L^2$-control problem in coefficients for a linear elliptic equation. II. Approximation of solutions and optimality conditions. Mathematical Control and Related Fields, 2016, 6 (4) : 595-628. doi: 10.3934/mcrf.2016017

[13]

Thierry Horsin, Peter I. Kogut. Optimal $L^2$-control problem in coefficients for a linear elliptic equation. I. Existence result. Mathematical Control and Related Fields, 2015, 5 (1) : 73-96. doi: 10.3934/mcrf.2015.5.73

[14]

Hanxiao Wang, Jingrui Sun, Jiongmin Yong. Weak closed-loop solvability of stochastic linear-quadratic optimal control problems. Discrete and Continuous Dynamical Systems, 2019, 39 (5) : 2785-2805. doi: 10.3934/dcds.2019117

[15]

Yadong Shu, Bo Li. Linear-quadratic optimal control for discrete-time stochastic descriptor systems. Journal of Industrial and Management Optimization, 2022, 18 (3) : 1583-1602. doi: 10.3934/jimo.2021034

[16]

Michael Herty, Lorenzo Pareschi, Sonja Steffensen. Mean--field control and Riccati equations. Networks and Heterogeneous Media, 2015, 10 (3) : 699-715. doi: 10.3934/nhm.2015.10.699

[17]

Yinggu Chen, Said HamadÈne, Tingshu Mu. Mean-field doubly reflected backward stochastic differential equations. Numerical Algebra, Control and Optimization, 2022  doi: 10.3934/naco.2022012

[18]

Galina Kurina, Sahlar Meherrem. Decomposition of discrete linear-quadratic optimal control problems for switching systems. Conference Publications, 2015, 2015 (special) : 764-774. doi: 10.3934/proc.2015.0764

[19]

Mohamed Aliane, Mohand Bentobache, Nacima Moussouni, Philippe Marthon. Direct method to solve linear-quadratic optimal control problems. Numerical Algebra, Control and Optimization, 2021, 11 (4) : 645-663. doi: 10.3934/naco.2021002

[20]

Diana Keller. Optimal control of a linear stochastic Schrödinger equation. Conference Publications, 2013, 2013 (special) : 437-446. doi: 10.3934/proc.2013.2013.437

2020 Impact Factor: 1.284

Metrics

  • PDF downloads (351)
  • HTML views (0)
  • Cited by (39)

Other articles
by authors

[Back to Top]