- Previous Article
- MCRF Home
- This Issue
-
Next Article
A non-exponential discounting time-inconsistent stochastic optimal control problem for jump-diffusion
Optimal control and zero-sum games for Markov chains of mean-field type
1. | Department of Mathematics, KTH Royal Institute of Technology, 100 44, Stockholm, Sweden |
2. | Learning & Game Theory Laboratory, New York University, 19 Washington Square North New York, NY 10011, USA |
We establish existence of Markov chains of mean-field type with unbounded jump intensities by means of a fixed point argument using the total variation distance. We further show existence of nearly-optimal controls and, using a Markov chain backward SDE approach, we suggest conditions for existence of an optimal control and a saddle-point for respectively a control problem and a zero-sum differential game associated with payoff functionals of mean-field type, under dynamics driven by such Markov chains of mean-field type.
References:
[1] |
V. E. Beneš, Existence of optimal stochastic control laws, SIAM J. Control, 9 1971,446-472.
doi: 10.1137/0309034. |
[2] |
P. Brèmaud, Point Processes and Queues: Martingale Dynamics, Springer-Verlag, New York-Berlin, 1981. |
[3] |
S. N. Cohen and R. J. Elliott,
Existence, uniqueness and comparisons for BSDEs in general spaces, Annals of Probability, 40 (2012), 2264-2297.
doi: 10.1214/11-AOP679. |
[4] |
S. N. Cohen and R. J. Elliott, Stochastic Calculus and Applications, Second edition. Probability and its Applications. Springer, Cham, 2015.
doi: 10.1007/978-1-4939-2867-5. |
[5] |
F. Confortola, M. Fuhrman and J. Jacod, Backward stochastic differential equations driven by a marked point process: an elementary approach, with an application to optimal control, Preprint, arXiv: 1407.0876, [math.PR], 2014. Google Scholar |
[6] |
S. E. Choutri and H. Tembine, A stochastic maximum principle for markov chains of mean-field type, Games, 9 (2018), Paper No. 84, 21 pp, https://doi.org/10.3390/g9040084.
doi: 10.3390/g9040084. |
[7] |
D. Dawson and X. Zheng, Law of large numbers and central limit theorem for unbounded jump mean-field models, Advances in Applied Mathematics, 12 (1991), 293-326.
doi: 10.1016/0196-8858(91)90015-B. |
[8] |
B. Djehiche and S. Hamadène, Optimal control and zero-sum stochastic differential game problems of mean-field type, Appl Math Optim., 2018, https://doi.org/10.1007/s00245-018-9525-6. Google Scholar |
[9] |
B. Djehiche and I. Kaj,
The rate function for some measure-valued jump processes, The Annals of Probability, 23 (1995), 1414-1438.
doi: 10.1214/aop/1176988190. |
[10] |
B. Djehiche and A. Schied,
Large deviations for hierarchical systems of interacting jump processes, Journal of Theoretical Probability, 11 (1998), 1-24.
doi: 10.1023/A:1021690707556. |
[11] |
I. Ekeland,
On the variational principle, J. Math. Anal. Appl., 47 (1974), 324-353.
doi: 10.1016/0022-247X(74)90025-0. |
[12] |
N. El Karoui and S. Hamadène,
BSDEs and risk-sensitive control, zero-sum and nonzero-sum game problems of stochastic functional differential equations, Stochastic Processes and their Application, 107 (2003), 145-169.
doi: 10.1016/S0304-4149(03)00059-0. |
[13] |
N. El Karoui, S. Peng and M.-C. Quenez,
Backward stochastic differential equations in finance, Mathematical Finance, 7 (1997), 1-71.
doi: 10.1111/1467-9965.00022. |
[14] |
R. J. Elliott and M. Kohlmann,
The variational principle and stochastic optimal control, Stochastics, 3 (1980), 229-241.
doi: 10.1080/17442508008833147. |
[15] |
S. N. Ethier and T. G. Kurtz, Markov Processes: Characterization and Convergence, John Wiley & Sons, Inc., New York, 1986.
doi: 10.1002/9780470316658. |
[16] |
S. Feng,
Large deviations for empirical process of mean-field interacting particle system with unbounded jumps, The Annals of Probability, 22 (1994), 2122-2151.
doi: 10.1214/aop/1176988496. |
[17] |
S. Feng and X. Zheng,
Solutions of a class of nonlinear master equations, Stochastic processes and their applications, 43 (1992), 65-84.
doi: 10.1016/0304-4149(92)90076-3. |
[18] |
S. Hamadène and J. P. Lepeltier, Backward equations, stochastic control and zero-sum stochastic differential games, Stochastics Stochastics Rep., 54 (1995), 221-231.
doi: 10.1080/17442509508834006. |
[19] |
B. Jourdain, S. Méléard and W. Woyczynski,
Nonlinear SDEs driven by Lévy processes and related PDEs, ALEA Lat. Am. J. Probab. Math. Stat., 4 (2008), 1-29.
|
[20] |
V. N. Kolokoltsov, Nonlinear Markov Processes and Kinetic Equations, Cambridge University Press, Cambridge, 2010.
doi: 10.1017/CBO9780511760303.![]() ![]() |
[21] |
V. N. Kolokoltsov, Nonlinear Markov games on a finite state space (mean-field and binary interactions), International Journal of Statistics and Probability, 1 (2012), 77. Google Scholar |
[22] |
C. Léonard, Some epidemic systems are long range interacting particle systems, Stochastic processes in epidemic systems (eds. J.P. Gabriel et al.), Lecture Notes in Biomathematics, volume 86, 1990, Springer. Google Scholar |
[23] |
C.Léonard, Large deviations for long range interacting particle systems with jumps, Annales de l'IHP Probabilités et Statistiques, 31 (1995), 289-323. |
[24] |
G. Nicolis and I. Prigogine, Self Organization in Non-Equilibrium Systems, New York-London-Sydney, 1977. |
[25] |
K. Oelschläger, A martingale approach to the law of large numbers for weakly interacting stochastic processes, The Annals of Probability, (1984), 458-479. Google Scholar |
[26] |
E. Pardoux and S. Peng, Adapted Solution of a Backward Stochastic Differential Equation, Systems and Control Letters, 14 (1990), 55-61.
doi: 10.1016/0167-6911(90)90082-6. |
[27] |
L. C. G. Rogers and D. Williams, Diffusions, Markov Processes and Martingales-Volume 2: Itô Calculus, Cambridge University Press, Cambridge, 2000.
doi: 10.1017/CBO9781107590120.![]() ![]() |
[28] |
F. Schlögl, Chemical reaction models for non-equilibrium phase transitions, Zeitschrift für Physik, 253 (1972), 147-161. Google Scholar |
[29] |
A. Sokol and N. R. Hansen, Exponential martingales and changes of measure for counting processes, Stochastic Analysis and Applications, 33 (2015), 823-843.
doi: 10.1080/07362994.2015.1040890. |
[30] |
A.-S. Sznitman, Topics in propagation of chaos, Ecole d'Été de Probabilités de Saint-Flour XIX 1989, 1964 (1991), 165-251.
doi: 10.1007/BFb0085169. |
show all references
References:
[1] |
V. E. Beneš, Existence of optimal stochastic control laws, SIAM J. Control, 9 1971,446-472.
doi: 10.1137/0309034. |
[2] |
P. Brèmaud, Point Processes and Queues: Martingale Dynamics, Springer-Verlag, New York-Berlin, 1981. |
[3] |
S. N. Cohen and R. J. Elliott,
Existence, uniqueness and comparisons for BSDEs in general spaces, Annals of Probability, 40 (2012), 2264-2297.
doi: 10.1214/11-AOP679. |
[4] |
S. N. Cohen and R. J. Elliott, Stochastic Calculus and Applications, Second edition. Probability and its Applications. Springer, Cham, 2015.
doi: 10.1007/978-1-4939-2867-5. |
[5] |
F. Confortola, M. Fuhrman and J. Jacod, Backward stochastic differential equations driven by a marked point process: an elementary approach, with an application to optimal control, Preprint, arXiv: 1407.0876, [math.PR], 2014. Google Scholar |
[6] |
S. E. Choutri and H. Tembine, A stochastic maximum principle for markov chains of mean-field type, Games, 9 (2018), Paper No. 84, 21 pp, https://doi.org/10.3390/g9040084.
doi: 10.3390/g9040084. |
[7] |
D. Dawson and X. Zheng, Law of large numbers and central limit theorem for unbounded jump mean-field models, Advances in Applied Mathematics, 12 (1991), 293-326.
doi: 10.1016/0196-8858(91)90015-B. |
[8] |
B. Djehiche and S. Hamadène, Optimal control and zero-sum stochastic differential game problems of mean-field type, Appl Math Optim., 2018, https://doi.org/10.1007/s00245-018-9525-6. Google Scholar |
[9] |
B. Djehiche and I. Kaj,
The rate function for some measure-valued jump processes, The Annals of Probability, 23 (1995), 1414-1438.
doi: 10.1214/aop/1176988190. |
[10] |
B. Djehiche and A. Schied,
Large deviations for hierarchical systems of interacting jump processes, Journal of Theoretical Probability, 11 (1998), 1-24.
doi: 10.1023/A:1021690707556. |
[11] |
I. Ekeland,
On the variational principle, J. Math. Anal. Appl., 47 (1974), 324-353.
doi: 10.1016/0022-247X(74)90025-0. |
[12] |
N. El Karoui and S. Hamadène,
BSDEs and risk-sensitive control, zero-sum and nonzero-sum game problems of stochastic functional differential equations, Stochastic Processes and their Application, 107 (2003), 145-169.
doi: 10.1016/S0304-4149(03)00059-0. |
[13] |
N. El Karoui, S. Peng and M.-C. Quenez,
Backward stochastic differential equations in finance, Mathematical Finance, 7 (1997), 1-71.
doi: 10.1111/1467-9965.00022. |
[14] |
R. J. Elliott and M. Kohlmann,
The variational principle and stochastic optimal control, Stochastics, 3 (1980), 229-241.
doi: 10.1080/17442508008833147. |
[15] |
S. N. Ethier and T. G. Kurtz, Markov Processes: Characterization and Convergence, John Wiley & Sons, Inc., New York, 1986.
doi: 10.1002/9780470316658. |
[16] |
S. Feng,
Large deviations for empirical process of mean-field interacting particle system with unbounded jumps, The Annals of Probability, 22 (1994), 2122-2151.
doi: 10.1214/aop/1176988496. |
[17] |
S. Feng and X. Zheng,
Solutions of a class of nonlinear master equations, Stochastic processes and their applications, 43 (1992), 65-84.
doi: 10.1016/0304-4149(92)90076-3. |
[18] |
S. Hamadène and J. P. Lepeltier, Backward equations, stochastic control and zero-sum stochastic differential games, Stochastics Stochastics Rep., 54 (1995), 221-231.
doi: 10.1080/17442509508834006. |
[19] |
B. Jourdain, S. Méléard and W. Woyczynski,
Nonlinear SDEs driven by Lévy processes and related PDEs, ALEA Lat. Am. J. Probab. Math. Stat., 4 (2008), 1-29.
|
[20] |
V. N. Kolokoltsov, Nonlinear Markov Processes and Kinetic Equations, Cambridge University Press, Cambridge, 2010.
doi: 10.1017/CBO9780511760303.![]() ![]() |
[21] |
V. N. Kolokoltsov, Nonlinear Markov games on a finite state space (mean-field and binary interactions), International Journal of Statistics and Probability, 1 (2012), 77. Google Scholar |
[22] |
C. Léonard, Some epidemic systems are long range interacting particle systems, Stochastic processes in epidemic systems (eds. J.P. Gabriel et al.), Lecture Notes in Biomathematics, volume 86, 1990, Springer. Google Scholar |
[23] |
C.Léonard, Large deviations for long range interacting particle systems with jumps, Annales de l'IHP Probabilités et Statistiques, 31 (1995), 289-323. |
[24] |
G. Nicolis and I. Prigogine, Self Organization in Non-Equilibrium Systems, New York-London-Sydney, 1977. |
[25] |
K. Oelschläger, A martingale approach to the law of large numbers for weakly interacting stochastic processes, The Annals of Probability, (1984), 458-479. Google Scholar |
[26] |
E. Pardoux and S. Peng, Adapted Solution of a Backward Stochastic Differential Equation, Systems and Control Letters, 14 (1990), 55-61.
doi: 10.1016/0167-6911(90)90082-6. |
[27] |
L. C. G. Rogers and D. Williams, Diffusions, Markov Processes and Martingales-Volume 2: Itô Calculus, Cambridge University Press, Cambridge, 2000.
doi: 10.1017/CBO9781107590120.![]() ![]() |
[28] |
F. Schlögl, Chemical reaction models for non-equilibrium phase transitions, Zeitschrift für Physik, 253 (1972), 147-161. Google Scholar |
[29] |
A. Sokol and N. R. Hansen, Exponential martingales and changes of measure for counting processes, Stochastic Analysis and Applications, 33 (2015), 823-843.
doi: 10.1080/07362994.2015.1040890. |
[30] |
A.-S. Sznitman, Topics in propagation of chaos, Ecole d'Été de Probabilités de Saint-Flour XIX 1989, 1964 (1991), 165-251.
doi: 10.1007/BFb0085169. |
[1] |
Bin Pei, Yong Xu, Yuzhen Bai. Convergence of p-th mean in an averaging principle for stochastic partial differential equations driven by fractional Brownian motion. Discrete & Continuous Dynamical Systems - B, 2020, 25 (3) : 1141-1158. doi: 10.3934/dcdsb.2019213 |
[2] |
Diana Keller. Optimal control of a linear stochastic Schrödinger equation. Conference Publications, 2013, 2013 (special) : 437-446. doi: 10.3934/proc.2013.2013.437 |
[3] |
Shanjian Tang, Fu Zhang. Path-dependent optimal stochastic control and viscosity solution of associated Bellman equations. Discrete & Continuous Dynamical Systems - A, 2015, 35 (11) : 5521-5553. doi: 10.3934/dcds.2015.35.5521 |
[4] |
J. Frédéric Bonnans, Justina Gianatti, Francisco J. Silva. On the convergence of the Sakawa-Shindo algorithm in stochastic control. Mathematical Control & Related Fields, 2016, 6 (3) : 391-406. doi: 10.3934/mcrf.2016008 |
[5] |
Paula A. González-Parra, Sunmi Lee, Leticia Velázquez, Carlos Castillo-Chavez. A note on the use of optimal control on a discrete time model of influenza dynamics. Mathematical Biosciences & Engineering, 2011, 8 (1) : 183-197. doi: 10.3934/mbe.2011.8.183 |
[6] |
Luke Finlay, Vladimir Gaitsgory, Ivan Lebedev. Linear programming solutions of periodic optimization problems: approximation of the optimal control. Journal of Industrial & Management Optimization, 2007, 3 (2) : 399-413. doi: 10.3934/jimo.2007.3.399 |
[7] |
Xiaohong Li, Mingxin Sun, Zhaohua Gong, Enmin Feng. Multistage optimal control for microbial fed-batch fermentation process. Journal of Industrial & Management Optimization, 2021 doi: 10.3934/jimo.2021040 |
[8] |
John T. Betts, Stephen Campbell, Claire Digirolamo. Examination of solving optimal control problems with delays using GPOPS-Ⅱ. Numerical Algebra, Control & Optimization, 2021, 11 (2) : 283-305. doi: 10.3934/naco.2020026 |
[9] |
David Cantala, Juan Sebastián Pereyra. Endogenous budget constraints in the assignment game. Journal of Dynamics & Games, 2015, 2 (3&4) : 207-225. doi: 10.3934/jdg.2015002 |
[10] |
Vakhtang Putkaradze, Stuart Rogers. Numerical simulations of a rolling ball robot actuated by internal point masses. Numerical Algebra, Control & Optimization, 2021, 11 (2) : 143-207. doi: 10.3934/naco.2020021 |
[11] |
Michael Grinfeld, Amy Novick-Cohen. Some remarks on stability for a phase field model with memory. Discrete & Continuous Dynamical Systems - A, 2006, 15 (4) : 1089-1117. doi: 10.3934/dcds.2006.15.1089 |
[12] |
Juliang Zhang, Jian Chen. Information sharing in a make-to-stock supply chain. Journal of Industrial & Management Optimization, 2014, 10 (4) : 1169-1189. doi: 10.3934/jimo.2014.10.1169 |
[13] |
Hyeong-Ohk Bae, Hyoungsuk So, Yeonghun Youn. Interior regularity to the steady incompressible shear thinning fluids with non-Standard growth. Networks & Heterogeneous Media, 2018, 13 (3) : 479-491. doi: 10.3934/nhm.2018021 |
[14] |
Yves Dumont, Frederic Chiroleu. Vector control for the Chikungunya disease. Mathematical Biosciences & Engineering, 2010, 7 (2) : 313-345. doi: 10.3934/mbe.2010.7.313 |
[15] |
Guido De Philippis, Antonio De Rosa, Jonas Hirsch. The area blow up set for bounded mean curvature submanifolds with respect to elliptic surface energy functionals. Discrete & Continuous Dynamical Systems - A, 2019, 39 (12) : 7031-7056. doi: 10.3934/dcds.2019243 |
[16] |
Y. Latushkin, B. Layton. The optimal gap condition for invariant manifolds. Discrete & Continuous Dynamical Systems - A, 1999, 5 (2) : 233-268. doi: 10.3934/dcds.1999.5.233 |
[17] |
Tomáš Roubíček. An energy-conserving time-discretisation scheme for poroelastic media with phase-field fracture emitting waves and heat. Discrete & Continuous Dynamical Systems - S, 2017, 10 (4) : 867-893. doi: 10.3934/dcdss.2017044 |
[18] |
Hakan Özadam, Ferruh Özbudak. A note on negacyclic and cyclic codes of length $p^s$ over a finite field of characteristic $p$. Advances in Mathematics of Communications, 2009, 3 (3) : 265-271. doi: 10.3934/amc.2009.3.265 |
[19] |
Wen-Bin Yang, Yan-Ling Li, Jianhua Wu, Hai-Xia Li. Dynamics of a food chain model with ratio-dependent and modified Leslie-Gower functional responses. Discrete & Continuous Dynamical Systems - B, 2015, 20 (7) : 2269-2290. doi: 10.3934/dcdsb.2015.20.2269 |
[20] |
Benrong Zheng, Xianpei Hong. Effects of take-back legislation on pricing and coordination in a closed-loop supply chain. Journal of Industrial & Management Optimization, 2021 doi: 10.3934/jimo.2021035 |
2019 Impact Factor: 0.857
Tools
Metrics
Other articles
by authors
[Back to Top]