# American Institute of Mathematical Sciences

January  2016, 3(1): 25-50. doi: 10.3934/jdg.2016002

## Discount-sensitive equilibria in zero-sum stochastic differential games

 1 Engineering Faculty, Universidad Veracruzana, Coatzacoalcos, Ver. 96538, Mexico

Received  April 2015 Revised  November 2015 Published  March 2016

We consider infinite-horizon zero-sum stochastic differential games with average payoff criteria, discount -sensitive criteria and, infinite-horizon undiscounted reward criteria which are sensitive to the growth rate of finite-horizon payoffs. These criteria include, average reward optimality, strong 0-discount optimality, strong -1-discount optimality, 0-discount optimality, bias optimality, F-strong average optimality and overtaking optimality. The main objective is to give conditions under which these criteria are interrelated.
Citation: Beatris A. Escobedo-Trujillo. Discount-sensitive equilibria in zero-sum stochastic differential games. Journal of Dynamics and Games, 2016, 3 (1) : 25-50. doi: 10.3934/jdg.2016002
##### References:
 [1] A. Arapostathis and V. Borkar, Uniform recurrence properties of controlled diffusions and applications to optimal control, SIAM J. Control Optim., 48 (2010), 4181-4223. doi: 10.1137/090762464. [2] A. Arapostathis, M. Ghosh and V. Borkar, Ergodic Control of Diffusion Processes, Vol. 143, Encyclopedia of Mathematics and its Applications, Cambridge University Press, 2012. [3] A. Arapostathis, V. Borkar and K. Surech, Relative value iteration for stochastic differential games, arXiv:1210.8188v2, Advances in Dynamic Games, 13 (2013), 3-27. doi: 10.1007/978-3-319-02690-9_1. [4] M. Bardi, Explicit solutions of some linear-quadratic mean field games. Networks and heterogeneous media, American Institute of Mathematical Sciences, 7 (2012), 243-261. doi: 10.3934/nhm.2012.7.243. [5] V. Borkar and M. Ghosh, Stochastic differential games: Occupation measure based approach, J. Optim. Theory Appl., 73 (1992), 359-385. Correction: 88 (1996), 251-252. doi: 10.1007/BF00940187. [6] R. Cavazos-Cadena and J. B. Lasserre, Strong 1-optimal stationary policies in denumerable Markov decision processes, Syst. Control Lett., 11 (1988), 65-71. doi: 10.1016/0167-6911(88)90113-2. [7] B. Escobedo-Trujillo, D. López-Barrientos and O. Hernández-Lerma, Bias and overtaking equilibria for zero-sum stochastic differential games, J. Optim. Theory Appl., 153 (2012), 662-687. doi: 10.1007/s10957-011-9974-4. [8] B. Escobedo-Trujillo and J. López-Barrientos, Nonzero-sum stochastic differential games with additive structure and average payoffs, Journal of Dynamics and Games, 1 (2014), 555-578. doi: 10.3934/jdg.2014.1.555. [9] K. Fan, Fixed-point and minimax theorems in locally convex linear spaces, Proc. N.A.S.U.S.A., 38 (1952), 121-126. doi: 10.1073/pnas.38.2.121. [10] A. Friedman, Stochastic Differential Equations and Applications, Vol. 1, Academic Press, New York, 1975. [11] J. Flynn, On optimality criteria for dynamic programs with long finite horizons, J. Math. Anal. Appl., 76 (1980), 202-208. doi: 10.1016/0022-247X(80)90072-4. [12] M. Ghosh, A. Arapostathis and S. Marcus, Ergodic control of switching diffusions, SIAM J. Control Optim., 35 (1997), 1952-1988. doi: 10.1137/S0363012996299302. [13] O. Hernández-Lerma and O. Vega-Amaya, Infinite-horizon Markov control processes with undiscounted cost criteria: from average to overtaking optimality, Appl. Math. (Warsaw), 25 (1998), 153-178. [14] O. Hernández-Lerma and J. Lasserre, Further Topics on Discrete-Time Markov Control Processes, Springer, New York, 1999. doi: 10.1007/978-1-4612-0561-6. [15] N. Hilgert and O. Hernández-Lerma, Bias optimality versus strong 0-discount optimality in Markov control processes with unbounded costs, Acta Applicandae Mathematicae, 77 (2003), 215-235. doi: 10.1023/A:1024996308133. [16] H. Jasso-Fuentes and O. Hernández-Lerma, Characterizations of overtaking optimality for controlled diffusion processes, Appl. Math. Optim., 57 (2008), 349-369. doi: 10.1007/s00245-007-9025-6. [17] H. Jasso-Fuentes and O. Hernández-Lerma, Ergodic control, bias, and sensitive discount optimality for Markov diffusion processes, Stochatic Analysis and Applications, 27 (2009), 363-385. doi: 10.1080/07362990802679034. [18] H. Jasso-Fuentes, J. López-Barrientos and B. Escobedo-Trujilo, Infinite horizon nonzero-sum stochastic differential games with additive structure, IMA J. Math. Control Inform. doi: 10.1093/imamci/dnv045, (2015). [19] S. Meyn and R. Tweedie, Stability of Markovian processes. III. Foster-Lyapunov criteria for continuous-time precesses, Adv. Appl. Prob., 25 (1993), 518-548. doi: 10.2307/1427522. [20] H. Morimoto and M. Ohashi, On linear stochastic differential games with average cost criterions, J. Optim. Theory Appl., 64 (1990), 127-140. doi: 10.1007/BF00940027. [21] A. Nowak, Sensitive equilibria for ergodic stochastic games with countable state spaces, Math. Meth. Oper. Res., 50 (1999), 65-76. [22] A. Nowak, Optimal strategies in a class of zero-sum ergodic stochastic games, Math. Meth. Oper. Res., 50 (1999), 399-419. doi: 10.1007/s001860050078. [23] A. Nowak, Remark on sensitive equilibria in stochastic games with additive reward and transition structure, Math. Meth. Oper. Res., 64 (2006), 481-494. doi: 10.1007/s00186-006-0090-4. [24] T. Prieto-Rumeau and O. Hernández-Lerma, Bias and overtaking equilibria for zero-sum continuous-time Markov games, Math. Meth. Oper. Res., 61 (2005), 437-454. doi: 10.1007/s001860400392. [25] T. Prieto-Rumeau and O. Hernández-Lerma, The Laurent series, sensitive discount and Blackwell optimality for continuous-time controlled Markov chains, Math. Meth. Oper. Res., 61 (2005), 123-145. doi: 10.1007/s001860400393. [26] M. Puterman, Sensitive discount optimality in controlled one-dimensional diffusions, Annals of Probability, 2 (1974), 408-419. doi: 10.1214/aop/1176996656. [27] W. Qingda and C. Xian, Strong average optimality criterion for continuos-time Markov decision processes, Kybernetika, 50 (2014), 950-977. [28] M. Sion, On general minimax theorems, Pacific J. Math., 8 (1958), 171-176. doi: 10.2140/pjm.1958.8.171. [29] W. Schmitendorf, Differential games without pure strategy saddle-point solutions, J. Optim. Theory Appl., 18 (1976), 81-92. doi: 10.1007/BF00933796. [30] Q. Zhu and X. Guo, Another set of conditions for strong $n$ ($n=-1,0$) discount optimality in Markov decision processes, Stochastic Analysis and Applications, 23 (2005), 953-974. doi: 10.1080/07362990500184865. [31] Q. Zhu, Bias optimality and strong $n$ ($n=-1,0$) discount optimality for Markov decision processes, J. Math. Anal. Appl., 334 (2007), 576-592. doi: 10.1016/j.jmaa.2007.01.002.

show all references

##### References:
 [1] A. Arapostathis and V. Borkar, Uniform recurrence properties of controlled diffusions and applications to optimal control, SIAM J. Control Optim., 48 (2010), 4181-4223. doi: 10.1137/090762464. [2] A. Arapostathis, M. Ghosh and V. Borkar, Ergodic Control of Diffusion Processes, Vol. 143, Encyclopedia of Mathematics and its Applications, Cambridge University Press, 2012. [3] A. Arapostathis, V. Borkar and K. Surech, Relative value iteration for stochastic differential games, arXiv:1210.8188v2, Advances in Dynamic Games, 13 (2013), 3-27. doi: 10.1007/978-3-319-02690-9_1. [4] M. Bardi, Explicit solutions of some linear-quadratic mean field games. Networks and heterogeneous media, American Institute of Mathematical Sciences, 7 (2012), 243-261. doi: 10.3934/nhm.2012.7.243. [5] V. Borkar and M. Ghosh, Stochastic differential games: Occupation measure based approach, J. Optim. Theory Appl., 73 (1992), 359-385. Correction: 88 (1996), 251-252. doi: 10.1007/BF00940187. [6] R. Cavazos-Cadena and J. B. Lasserre, Strong 1-optimal stationary policies in denumerable Markov decision processes, Syst. Control Lett., 11 (1988), 65-71. doi: 10.1016/0167-6911(88)90113-2. [7] B. Escobedo-Trujillo, D. López-Barrientos and O. Hernández-Lerma, Bias and overtaking equilibria for zero-sum stochastic differential games, J. Optim. Theory Appl., 153 (2012), 662-687. doi: 10.1007/s10957-011-9974-4. [8] B. Escobedo-Trujillo and J. López-Barrientos, Nonzero-sum stochastic differential games with additive structure and average payoffs, Journal of Dynamics and Games, 1 (2014), 555-578. doi: 10.3934/jdg.2014.1.555. [9] K. Fan, Fixed-point and minimax theorems in locally convex linear spaces, Proc. N.A.S.U.S.A., 38 (1952), 121-126. doi: 10.1073/pnas.38.2.121. [10] A. Friedman, Stochastic Differential Equations and Applications, Vol. 1, Academic Press, New York, 1975. [11] J. Flynn, On optimality criteria for dynamic programs with long finite horizons, J. Math. Anal. Appl., 76 (1980), 202-208. doi: 10.1016/0022-247X(80)90072-4. [12] M. Ghosh, A. Arapostathis and S. Marcus, Ergodic control of switching diffusions, SIAM J. Control Optim., 35 (1997), 1952-1988. doi: 10.1137/S0363012996299302. [13] O. Hernández-Lerma and O. Vega-Amaya, Infinite-horizon Markov control processes with undiscounted cost criteria: from average to overtaking optimality, Appl. Math. (Warsaw), 25 (1998), 153-178. [14] O. Hernández-Lerma and J. Lasserre, Further Topics on Discrete-Time Markov Control Processes, Springer, New York, 1999. doi: 10.1007/978-1-4612-0561-6. [15] N. Hilgert and O. Hernández-Lerma, Bias optimality versus strong 0-discount optimality in Markov control processes with unbounded costs, Acta Applicandae Mathematicae, 77 (2003), 215-235. doi: 10.1023/A:1024996308133. [16] H. Jasso-Fuentes and O. Hernández-Lerma, Characterizations of overtaking optimality for controlled diffusion processes, Appl. Math. Optim., 57 (2008), 349-369. doi: 10.1007/s00245-007-9025-6. [17] H. Jasso-Fuentes and O. Hernández-Lerma, Ergodic control, bias, and sensitive discount optimality for Markov diffusion processes, Stochatic Analysis and Applications, 27 (2009), 363-385. doi: 10.1080/07362990802679034. [18] H. Jasso-Fuentes, J. López-Barrientos and B. Escobedo-Trujilo, Infinite horizon nonzero-sum stochastic differential games with additive structure, IMA J. Math. Control Inform. doi: 10.1093/imamci/dnv045, (2015). [19] S. Meyn and R. Tweedie, Stability of Markovian processes. III. Foster-Lyapunov criteria for continuous-time precesses, Adv. Appl. Prob., 25 (1993), 518-548. doi: 10.2307/1427522. [20] H. Morimoto and M. Ohashi, On linear stochastic differential games with average cost criterions, J. Optim. Theory Appl., 64 (1990), 127-140. doi: 10.1007/BF00940027. [21] A. Nowak, Sensitive equilibria for ergodic stochastic games with countable state spaces, Math. Meth. Oper. Res., 50 (1999), 65-76. [22] A. Nowak, Optimal strategies in a class of zero-sum ergodic stochastic games, Math. Meth. Oper. Res., 50 (1999), 399-419. doi: 10.1007/s001860050078. [23] A. Nowak, Remark on sensitive equilibria in stochastic games with additive reward and transition structure, Math. Meth. Oper. Res., 64 (2006), 481-494. doi: 10.1007/s00186-006-0090-4. [24] T. Prieto-Rumeau and O. Hernández-Lerma, Bias and overtaking equilibria for zero-sum continuous-time Markov games, Math. Meth. Oper. Res., 61 (2005), 437-454. doi: 10.1007/s001860400392. [25] T. Prieto-Rumeau and O. Hernández-Lerma, The Laurent series, sensitive discount and Blackwell optimality for continuous-time controlled Markov chains, Math. Meth. Oper. Res., 61 (2005), 123-145. doi: 10.1007/s001860400393. [26] M. Puterman, Sensitive discount optimality in controlled one-dimensional diffusions, Annals of Probability, 2 (1974), 408-419. doi: 10.1214/aop/1176996656. [27] W. Qingda and C. Xian, Strong average optimality criterion for continuos-time Markov decision processes, Kybernetika, 50 (2014), 950-977. [28] M. Sion, On general minimax theorems, Pacific J. Math., 8 (1958), 171-176. doi: 10.2140/pjm.1958.8.171. [29] W. Schmitendorf, Differential games without pure strategy saddle-point solutions, J. Optim. Theory Appl., 18 (1976), 81-92. doi: 10.1007/BF00933796. [30] Q. Zhu and X. Guo, Another set of conditions for strong $n$ ($n=-1,0$) discount optimality in Markov decision processes, Stochastic Analysis and Applications, 23 (2005), 953-974. doi: 10.1080/07362990500184865. [31] Q. Zhu, Bias optimality and strong $n$ ($n=-1,0$) discount optimality for Markov decision processes, J. Math. Anal. Appl., 334 (2007), 576-592. doi: 10.1016/j.jmaa.2007.01.002.
 [1] Beatris Adriana Escobedo-Trujillo, Alejandro Alaffita-Hernández, Raquiel López-Martínez. Constrained stochastic differential games with additive structure: Average and discount payoffs. Journal of Dynamics and Games, 2018, 5 (2) : 109-141. doi: 10.3934/jdg.2018008 [2] Beatris Adriana Escobedo-Trujillo, José Daniel López-Barrientos. Nonzero-sum stochastic differential games with additive structure and average payoffs. Journal of Dynamics and Games, 2014, 1 (4) : 555-578. doi: 10.3934/jdg.2014.1.555 [3] Yannick Viossat. Game dynamics and Nash equilibria. Journal of Dynamics and Games, 2014, 1 (3) : 537-553. doi: 10.3934/jdg.2014.1.537 [4] Sheri M. Markose. Complex type 4 structure changing dynamics of digital agents: Nash equilibria of a game with arms race in innovations. Journal of Dynamics and Games, 2017, 4 (3) : 255-284. doi: 10.3934/jdg.2017015 [5] Piernicola Bettiol, Nathalie Khalil. Necessary optimality conditions for average cost minimization problems. Discrete and Continuous Dynamical Systems - B, 2019, 24 (5) : 2093-2124. doi: 10.3934/dcdsb.2019086 [6] Margarida Carvalho, João Pedro Pedroso, João Saraiva. Electricity day-ahead markets: Computation of Nash equilibria. Journal of Industrial and Management Optimization, 2015, 11 (3) : 985-998. doi: 10.3934/jimo.2015.11.985 [7] Athanasios Kehagias. A note on the Nash equilibria of some multi-player reachability/safety games. Journal of Dynamics and Games, 2022, 9 (1) : 117-122. doi: 10.3934/jdg.2021028 [8] Matthew Bourque, T. E. S. Raghavan. Policy improvement for perfect information additive reward and additive transition stochastic games with discounted and average payoffs. Journal of Dynamics and Games, 2014, 1 (3) : 347-361. doi: 10.3934/jdg.2014.1.347 [9] Mei Ju Luo, Yi Zeng Chen. Smoothing and sample average approximation methods for solving stochastic generalized Nash equilibrium problems. Journal of Industrial and Management Optimization, 2016, 12 (1) : 1-15. doi: 10.3934/jimo.2016.12.1 [10] Alejandra Fonseca-Morales, Onésimo Hernández-Lerma. A note on differential games with Pareto-optimal NASH equilibria: Deterministic and stochastic models†. Journal of Dynamics and Games, 2017, 4 (3) : 195-203. doi: 10.3934/jdg.2017012 [11] Adela Capătă. Optimality conditions for strong vector equilibrium problems under a weak constraint qualification. Journal of Industrial and Management Optimization, 2015, 11 (2) : 563-574. doi: 10.3934/jimo.2015.11.563 [12] Tieliang Gong, Chen Xu, Hong Chen. Modal additive models with data-driven structure identification. Mathematical Foundations of Computing, 2020, 3 (3) : 165-183. doi: 10.3934/mfc.2020016 [13] Zilong Wang, Guang Gong, Rongquan Feng. A generalized construction of OFDM $M$-QAM sequences with low peak-to-average power ratio. Advances in Mathematics of Communications, 2009, 3 (4) : 421-428. doi: 10.3934/amc.2009.3.421 [14] Feimin Zhong, Jinxing Xie, Yuwei Shen. Bargaining in a multi-echelon supply chain with power structure: KS solution vs. Nash solution. Journal of Industrial and Management Optimization, 2022, 18 (1) : 635-654. doi: 10.3934/jimo.2020172 [15] Jean-Jérôme Casanova. Existence of time-periodic strong solutions to a fluid–structure system. Discrete and Continuous Dynamical Systems, 2019, 39 (6) : 3291-3313. doi: 10.3934/dcds.2019136 [16] Pengmiao Hao, Xuechen Wang, Junjie Wei. Global Hopf bifurcation of a population model with stage structure and strong Allee effect. Discrete and Continuous Dynamical Systems - S, 2017, 10 (5) : 973-993. doi: 10.3934/dcdss.2017051 [17] Heinz Schättler, Urszula Ledzewicz, Helmut Maurer. Sufficient conditions for strong local optimality in optimal control problems with $L_{2}$-type objectives and control constraints. Discrete and Continuous Dynamical Systems - B, 2014, 19 (8) : 2657-2679. doi: 10.3934/dcdsb.2014.19.2657 [18] Roghayeh Mohammadi Hesari, Mahboubeh Hosseinabadi, Rashid Rezaei, Karim Samei. $\mathbb{F}_{p^{m}}\mathbb{F}_{p^{m}}{[u^2]}$-additive skew cyclic codes of length $2p^s$. Advances in Mathematics of Communications, 2022  doi: 10.3934/amc.2022023 [19] Haitao Che, Haibin Chen, Guanglu Zhou. New M-eigenvalue intervals and application to the strong ellipticity of fourth-order partially symmetric tensors. Journal of Industrial and Management Optimization, 2021, 17 (6) : 3685-3694. doi: 10.3934/jimo.2020139 [20] Chong Wang, Gang Wang, Lixia Liu. Sharp bounds on the minimum $M$-eigenvalue and strong ellipticity condition of elasticity $Z$-tensors-tensors. Journal of Industrial and Management Optimization, 2021  doi: 10.3934/jimo.2021205

Impact Factor: