April  2018, 5(2): 109-141. doi: 10.3934/jdg.2018008

Constrained stochastic differential games with additive structure: Average and discount payoffs

1. 

Engineering Faculty. Universidad Veracruzana, Coatzacoalcos, Ver., México

2. 

Mathematics Faculty, Universidad Veracruzana, Xalapa, Ver., México

* Corresponding author: Beatris Adriana Escobedo-Trujillo

Received  July 2017 Revised  December 2017 Published  February 2018

This paper deals with two-person nonzero-sum stochastic differential games (SDGs) with an additive structure, subject to constraints that are additive also. Our main objective is to give conditions for the existence of constrained Nash equilibria for the case of infinite-horizon discounted payoff. This is done by means of the Lagrange multipliers approach combined with dynamic programming arguments. Then, following the vanishing discount approach, the results in the discounted case are used to obtain constrained Nash equilibria in the case of long-run average payoff.

Citation: Beatris Adriana Escobedo-Trujillo, Alejandro Alaffita-Hernández, Raquiel López-Martínez. Constrained stochastic differential games with additive structure: Average and discount payoffs. Journal of Dynamics and Games, 2018, 5 (2) : 109-141. doi: 10.3934/jdg.2018008
References:
[1]

R. Akella and P. Kumar, Optimal control of production rate in a failure prone manufacturing system, IEEE Trans. Automatic Control., 31 (1986), 116-126.  doi: 10.1109/TAC.1986.1104206.

[2]

E. Altman and A. Shwartz A, Constrained Markov games: Nash equilibria, in: J.A. Filar, V. Gaitsgory, K. Mizukami (Eds.), Advances in Dynamic Games and Applications, Birkhäuser, Boston, 5 (2000), 213-221.

[3]

E. AltmanK. AvrachenkovR. Marquez and G. Miller, Zero-sum constrained stochastic games with independent state processes, Math. Meth. Oper. Res., 62 (2005), 375-386.  doi: 10.1007/s00186-005-0034-4.

[4]

J. Alvarez-Mena and O. Hernández-Lerma, Existence of Nash equilibria for constrained stochastic games, Math. Methods Oper. Res., 63 (2006), 261-285.  doi: 10.1007/s00186-005-0003-y.

[5]

A. Arapostathis and V. Borkar, Uniform recurrence properties of controlled diffusions and applications to optimal control, SIAM J. Control Optim., 48 (2010), 4181-4223.  doi: 10.1137/090762464.

[6]

V. BogachevN. Krylov and M. Röckner, Regularity and global bounds for the densities of invariant measures of diffusion processe, Dokl. Akad. Nauk., 405 (2005), 583-587. 

[7]

V. BogachevM. Röckner and V. Stannat, Uniqueness of solutions of elliptic equations and uniqueness pf invariant measures of diffusions processes, Sb. Math., 193 (2002), 945-976. 

[8]

V. Borkar, A topology for Markov controls, Appl. Math. Optim., 20 (1989), 55-62.  doi: 10.1007/BF01447645.

[9]

V. Borkar and M. Ghosh, Controlled diffusions with constraints, J. Math. Anal. Appl., 152 (1990), 88-108.  doi: 10.1016/0022-247X(90)90094-V.

[10]

V. Borkar and M. Ghosh, Controlled diffusions with constraints Ⅱ, J. Math. Anal. Appl., 176 (1993), 310-321.  doi: 10.1006/jmaa.1993.1216.

[11]

V. Borkar and M. Ghosh, Stochastic differential games: Occupation measure based approach, J. Optim. Theory Appl., 73 (1992), 359-385. 

[12]

A. Calderón and J. Rosenblueth, Minimizing approximate original solutions for commensurate delayed controls, Appl. Math. Lett., 7 (1994), 5-10.  doi: 10.1016/0893-9659(94)90063-9.

[13]

A. DvoretzkyA. Wald and J. Wolfowitz, Elimination of randomization in certain statistical decision procedure and zero-sum two person games, Ann. Math. Statist., 22 (1951), 1-21.  doi: 10.1214/aoms/1177729689.

[14]

B. Escobedo-TrujilloJ. López-Barrientos and O. Hernández-Lerma, Bias and overtaking equilibria for zero-sum stochastic differential games, J. Optim. Theory Appl., 153 (2012), 662-687.  doi: 10.1007/s10957-011-9974-4.

[15]

G. Folland, Real Analysis. Modern Techniques and Their Applications, 2$^{nd}$ edition, John Wiley and Sons, New York, 1999.

[16]

M. GhoshA. Arapostathis and S. Marcus, Ergodic control of switching diffusions, SIAM J. Control Optim., 35 (1997), 1962-1988.  doi: 10.1137/S0363012996299302.

[17]

M. Ghosh and A. Bagchi, Stochastic game with average payoff criterion, Appl. Math. Optim., 38 (1998), 283-301.  doi: 10.1007/s002459900092.

[18]

D. Gilbarg and N. Trudinger, Elliptic Partial Differential Equations of Second Order. Reprinted version, Springer-Verlag, Berlin, 2001.

[19]

P. Glynn and S. Meyn, A Liapounov bound for solutions of the Poisson equation, Ann. Prob., 24 (1996), 916-931.  doi: 10.1214/aop/1039639370.

[20]

H. Hernández-Hernández, Existence of nash equilibria in discounted nonzero-sum stochastic games with additive structure, Morfismos, 6 (2002), 43-65. 

[21]

H. Jasso-Fuentes and O. Hernández-Lerma, Characterizations of overtaking optimality for controlled diffusion processes, Appl. Math. Optim., 57 (2008), 349-369.  doi: 10.1007/s00245-007-9025-6.

[22]

H. Jasso-Fuentes and G. Yin, Advanced Criteria for Controlled Markov-codulated Diffusions in an Infinite Horizon: Overtaking, Bias, and Blackwell Optimality, Science Press, Beijing China, 2013.

[23]

H. Jasso-FuentesB. Escobedo-Trujilo and A. Mendoza-Pérez, The Lagrange and the vanishing discount techniques to controlled diffusions with cost constraints, J. Math. Anal. Appl., 437 (2016), 999-1035.  doi: 10.1016/j.jmaa.2016.01.036.

[24]

H. Jasso-FuentesJ. López-Barrientos and B. Escobedo-Trujilo, Infinite horizon nonzero-sum stochastic differential games with additive structure, IMA J. Math. Control Inform., 34 (2017), 283-309.  doi: 10.1093/imamci/dnv045.

[25]

F. Klebaner, Introduction to Stochastic Calculus with Applications, 2$^{nd}$ edition, Imperial College Press, London, 2005.

[26]

H. Küenle, Equilibrium strategies in stochastic games with additive cost and transition structure, Internat. Game Theory Rev., 1 (1999), 131-147.  doi: 10.1142/S0219198999000098.

[27]

A. Mendoza-PérezH. Jasso-Fuentes and O. Hernández, The Lagrange approach to ergodic control of diffusions with cost constraints, Optimization, 64 (2015), 179-196.  doi: 10.1080/02331934.2012.736992.

[28]

S. Meyn and R. Tweedie, Stability of Markovian processes, Ⅲ. Foster-Lyapunov criteria for continuous-time precesses, Adv. Appl. Prob., 25 (1993), 518-548. 

[29]

A. Nowak, Remark on sensitive equilibria in stochastic games with additive reward and transition structure, Math. Meth. Oper. Res., 64 (2006), 481-494.  doi: 10.1007/s00186-006-0090-4.

[30]

B. /Oksendal, Stochastic Differential Equations: An Introduction with Applications, 4$^{th}$ edition, Springer-Verlag, New York, 1994.

[31]

T. Prieto-Rumeau and O. Hernández-Lerma, Bias and overtaking equilibria for zero-sum continuous-time Markov games, Math. Meth. Oper. Res., 61 (2005), 437-454.  doi: 10.1007/s001860400392.

[32]

T. RaghavanS. Tijs and O. Vrieze, On stochastic games with additive reward and transition structure, J. Optim. Theory Appl., 47 (1985), 451-464.  doi: 10.1007/BF00942191.

[33]

V. Singh and N. Hemachandra, A characterization of stationary Nash equilibria of constrained stochastic games with independent state processes, Operations Research Letters, 42 (2014), 48-52.  doi: 10.1016/j.orl.2013.11.007.

[34]

J. Warga, Optimal Control of Differential and Functional Equations, Academic Press, New York, 1972.

[35]

Q. Wei and X. Chen, Constrained stochastic games with the average payoff criteria, Operations Research Letters, 43 (2015), 83-88.  doi: 10.1016/j.orl.2014.12.003.

show all references

References:
[1]

R. Akella and P. Kumar, Optimal control of production rate in a failure prone manufacturing system, IEEE Trans. Automatic Control., 31 (1986), 116-126.  doi: 10.1109/TAC.1986.1104206.

[2]

E. Altman and A. Shwartz A, Constrained Markov games: Nash equilibria, in: J.A. Filar, V. Gaitsgory, K. Mizukami (Eds.), Advances in Dynamic Games and Applications, Birkhäuser, Boston, 5 (2000), 213-221.

[3]

E. AltmanK. AvrachenkovR. Marquez and G. Miller, Zero-sum constrained stochastic games with independent state processes, Math. Meth. Oper. Res., 62 (2005), 375-386.  doi: 10.1007/s00186-005-0034-4.

[4]

J. Alvarez-Mena and O. Hernández-Lerma, Existence of Nash equilibria for constrained stochastic games, Math. Methods Oper. Res., 63 (2006), 261-285.  doi: 10.1007/s00186-005-0003-y.

[5]

A. Arapostathis and V. Borkar, Uniform recurrence properties of controlled diffusions and applications to optimal control, SIAM J. Control Optim., 48 (2010), 4181-4223.  doi: 10.1137/090762464.

[6]

V. BogachevN. Krylov and M. Röckner, Regularity and global bounds for the densities of invariant measures of diffusion processe, Dokl. Akad. Nauk., 405 (2005), 583-587. 

[7]

V. BogachevM. Röckner and V. Stannat, Uniqueness of solutions of elliptic equations and uniqueness pf invariant measures of diffusions processes, Sb. Math., 193 (2002), 945-976. 

[8]

V. Borkar, A topology for Markov controls, Appl. Math. Optim., 20 (1989), 55-62.  doi: 10.1007/BF01447645.

[9]

V. Borkar and M. Ghosh, Controlled diffusions with constraints, J. Math. Anal. Appl., 152 (1990), 88-108.  doi: 10.1016/0022-247X(90)90094-V.

[10]

V. Borkar and M. Ghosh, Controlled diffusions with constraints Ⅱ, J. Math. Anal. Appl., 176 (1993), 310-321.  doi: 10.1006/jmaa.1993.1216.

[11]

V. Borkar and M. Ghosh, Stochastic differential games: Occupation measure based approach, J. Optim. Theory Appl., 73 (1992), 359-385. 

[12]

A. Calderón and J. Rosenblueth, Minimizing approximate original solutions for commensurate delayed controls, Appl. Math. Lett., 7 (1994), 5-10.  doi: 10.1016/0893-9659(94)90063-9.

[13]

A. DvoretzkyA. Wald and J. Wolfowitz, Elimination of randomization in certain statistical decision procedure and zero-sum two person games, Ann. Math. Statist., 22 (1951), 1-21.  doi: 10.1214/aoms/1177729689.

[14]

B. Escobedo-TrujilloJ. López-Barrientos and O. Hernández-Lerma, Bias and overtaking equilibria for zero-sum stochastic differential games, J. Optim. Theory Appl., 153 (2012), 662-687.  doi: 10.1007/s10957-011-9974-4.

[15]

G. Folland, Real Analysis. Modern Techniques and Their Applications, 2$^{nd}$ edition, John Wiley and Sons, New York, 1999.

[16]

M. GhoshA. Arapostathis and S. Marcus, Ergodic control of switching diffusions, SIAM J. Control Optim., 35 (1997), 1962-1988.  doi: 10.1137/S0363012996299302.

[17]

M. Ghosh and A. Bagchi, Stochastic game with average payoff criterion, Appl. Math. Optim., 38 (1998), 283-301.  doi: 10.1007/s002459900092.

[18]

D. Gilbarg and N. Trudinger, Elliptic Partial Differential Equations of Second Order. Reprinted version, Springer-Verlag, Berlin, 2001.

[19]

P. Glynn and S. Meyn, A Liapounov bound for solutions of the Poisson equation, Ann. Prob., 24 (1996), 916-931.  doi: 10.1214/aop/1039639370.

[20]

H. Hernández-Hernández, Existence of nash equilibria in discounted nonzero-sum stochastic games with additive structure, Morfismos, 6 (2002), 43-65. 

[21]

H. Jasso-Fuentes and O. Hernández-Lerma, Characterizations of overtaking optimality for controlled diffusion processes, Appl. Math. Optim., 57 (2008), 349-369.  doi: 10.1007/s00245-007-9025-6.

[22]

H. Jasso-Fuentes and G. Yin, Advanced Criteria for Controlled Markov-codulated Diffusions in an Infinite Horizon: Overtaking, Bias, and Blackwell Optimality, Science Press, Beijing China, 2013.

[23]

H. Jasso-FuentesB. Escobedo-Trujilo and A. Mendoza-Pérez, The Lagrange and the vanishing discount techniques to controlled diffusions with cost constraints, J. Math. Anal. Appl., 437 (2016), 999-1035.  doi: 10.1016/j.jmaa.2016.01.036.

[24]

H. Jasso-FuentesJ. López-Barrientos and B. Escobedo-Trujilo, Infinite horizon nonzero-sum stochastic differential games with additive structure, IMA J. Math. Control Inform., 34 (2017), 283-309.  doi: 10.1093/imamci/dnv045.

[25]

F. Klebaner, Introduction to Stochastic Calculus with Applications, 2$^{nd}$ edition, Imperial College Press, London, 2005.

[26]

H. Küenle, Equilibrium strategies in stochastic games with additive cost and transition structure, Internat. Game Theory Rev., 1 (1999), 131-147.  doi: 10.1142/S0219198999000098.

[27]

A. Mendoza-PérezH. Jasso-Fuentes and O. Hernández, The Lagrange approach to ergodic control of diffusions with cost constraints, Optimization, 64 (2015), 179-196.  doi: 10.1080/02331934.2012.736992.

[28]

S. Meyn and R. Tweedie, Stability of Markovian processes, Ⅲ. Foster-Lyapunov criteria for continuous-time precesses, Adv. Appl. Prob., 25 (1993), 518-548. 

[29]

A. Nowak, Remark on sensitive equilibria in stochastic games with additive reward and transition structure, Math. Meth. Oper. Res., 64 (2006), 481-494.  doi: 10.1007/s00186-006-0090-4.

[30]

B. /Oksendal, Stochastic Differential Equations: An Introduction with Applications, 4$^{th}$ edition, Springer-Verlag, New York, 1994.

[31]

T. Prieto-Rumeau and O. Hernández-Lerma, Bias and overtaking equilibria for zero-sum continuous-time Markov games, Math. Meth. Oper. Res., 61 (2005), 437-454.  doi: 10.1007/s001860400392.

[32]

T. RaghavanS. Tijs and O. Vrieze, On stochastic games with additive reward and transition structure, J. Optim. Theory Appl., 47 (1985), 451-464.  doi: 10.1007/BF00942191.

[33]

V. Singh and N. Hemachandra, A characterization of stationary Nash equilibria of constrained stochastic games with independent state processes, Operations Research Letters, 42 (2014), 48-52.  doi: 10.1016/j.orl.2013.11.007.

[34]

J. Warga, Optimal Control of Differential and Functional Equations, Academic Press, New York, 1972.

[35]

Q. Wei and X. Chen, Constrained stochastic games with the average payoff criteria, Operations Research Letters, 43 (2015), 83-88.  doi: 10.1016/j.orl.2014.12.003.

[1]

Takeshi Fukao, Nobuyuki Kenmochi. Abstract theory of variational inequalities and Lagrange multipliers. Conference Publications, 2013, 2013 (special) : 237-246. doi: 10.3934/proc.2013.2013.237

[2]

Karla L. Cortez, Javier F. Rosenblueth. Normality and uniqueness of Lagrange multipliers. Discrete and Continuous Dynamical Systems, 2018, 38 (6) : 3169-3188. doi: 10.3934/dcds.2018138

[3]

Tao Jie, Gao Yan. Computing shadow prices with multiple Lagrange multipliers. Journal of Industrial and Management Optimization, 2021, 17 (5) : 2307-2329. doi: 10.3934/jimo.2020070

[4]

Maria do Rosário de Pinho, Ilya Shvartsman. Lipschitz continuity of optimal control and Lagrange multipliers in a problem with mixed and pure state constraints. Discrete and Continuous Dynamical Systems, 2011, 29 (2) : 505-522. doi: 10.3934/dcds.2011.29.505

[5]

Ana Bela Cruzeiro. Navier-Stokes and stochastic Navier-Stokes equations via Lagrange multipliers. Journal of Geometric Mechanics, 2019, 11 (4) : 553-560. doi: 10.3934/jgm.2019027

[6]

Ying Gao, Xinmin Yang, Jin Yang, Hong Yan. Scalarizations and Lagrange multipliers for approximate solutions in the vector optimization problems with set-valued maps. Journal of Industrial and Management Optimization, 2015, 11 (2) : 673-683. doi: 10.3934/jimo.2015.11.673

[7]

Manfred Deistler. Singular arma systems: A structure theory. Numerical Algebra, Control and Optimization, 2019, 9 (3) : 383-391. doi: 10.3934/naco.2019025

[8]

Beatris Adriana Escobedo-Trujillo, José Daniel López-Barrientos. Nonzero-sum stochastic differential games with additive structure and average payoffs. Journal of Dynamics and Games, 2014, 1 (4) : 555-578. doi: 10.3934/jdg.2014.1.555

[9]

Tieliang Gong, Chen Xu, Hong Chen. Modal additive models with data-driven structure identification. Mathematical Foundations of Computing, 2020, 3 (3) : 165-183. doi: 10.3934/mfc.2020016

[10]

Liming Wang. A passivity-based stability criterion for reaction diffusion systems with interconnected structure. Discrete and Continuous Dynamical Systems - B, 2012, 17 (1) : 303-323. doi: 10.3934/dcdsb.2012.17.303

[11]

Ting-Hao Hsu, Gail S. K. Wolkowicz. A criterion for the existence of relaxation oscillations with applications to predator-prey systems and an epidemic model. Discrete and Continuous Dynamical Systems - B, 2020, 25 (4) : 1257-1277. doi: 10.3934/dcdsb.2019219

[12]

Georg Ostrovski, Sebastian van Strien. Payoff performance of fictitious play. Journal of Dynamics and Games, 2014, 1 (4) : 621-638. doi: 10.3934/jdg.2014.1.621

[13]

Henry Jacobs, Joris Vankerschaver. Fluid-structure interaction in the Lagrange-Poincaré formalism: The Navier-Stokes and inviscid regimes. Journal of Geometric Mechanics, 2014, 6 (1) : 39-66. doi: 10.3934/jgm.2014.6.39

[14]

Lasse Kliemann, Elmira Shirazi Sheykhdarabadi, Anand Srivastav. Price of anarchy for graph coloring games with concave payoff. Journal of Dynamics and Games, 2017, 4 (1) : 41-58. doi: 10.3934/jdg.2017003

[15]

D. Warren, K Najarian. Learning theory applied to Sigmoid network classification of protein biological function using primary protein structure. Conference Publications, 2003, 2003 (Special) : 898-904. doi: 10.3934/proc.2003.2003.898

[16]

Andrea Davini, Lin Wang. On the vanishing discount problem from the negative direction. Discrete and Continuous Dynamical Systems, 2021, 41 (5) : 2377-2389. doi: 10.3934/dcds.2020368

[17]

Simon Hoof. Cooperative dynamic advertising via state-dependent payoff weights. Journal of Dynamics and Games, 2019, 6 (3) : 195-209. doi: 10.3934/jdg.2019014

[18]

Richard Sharp, Anastasios Stylianou. Statistics of multipliers for hyperbolic rational maps. Discrete and Continuous Dynamical Systems, 2022, 42 (3) : 1225-1241. doi: 10.3934/dcds.2021153

[19]

Pedro L. García, Antonio Fernández, César Rodrigo. Variational integrators for discrete Lagrange problems. Journal of Geometric Mechanics, 2010, 2 (4) : 343-374. doi: 10.3934/jgm.2010.2.343

[20]

Giovanni Bonfanti, Arrigo Cellina. The validity of the Euler-Lagrange equation. Discrete and Continuous Dynamical Systems, 2010, 28 (2) : 511-517. doi: 10.3934/dcds.2010.28.511

[Back to Top]