Limit value for optimal control with general means

Xiaoxi  Li; Marc  Quincampoix; Jérôme Renault

doi:10.3934/dcds.2016.36.2113

Article Contents

2016, Volume 36, Issue 4: 2113-2132. Doi: 10.3934/dcds.2016.36.2113

This issue Previous Article Quartic Julia sets including any two copies of quadratic Julia sets Next Article Stability analysis of reaction-diffusion models on evolving domains: The effects of cross-diffusion

Limit value for optimal control with general means

1.
Sorbonne Universités, UPMC Univ Paris 06, IMJ-PRG, UMR 7586, CNRS, Univ Paris Diderot, Sorbonne Paris Cité, 4 Place Jussieu, 75005 Paris
2.
Laboratoire de Mathématiques de Bretagne Atlantique, UMR 6205, Université de Brest, 6 Avenue Victor Le Gorgeu, 29200 Brest
3.
TSE (GREMAQ, Université Toulouse 1 Capitole and GDR 2932 Théorie des Jeux), 21 allée de Brienne, 31000 Toulouse

Received: February 28, 2015

Revised: June 30, 2015

Published: May 2017

Abstract Related Papers Cited by

Abstract

We consider optimal control problems where the running cost of the trajectory is evaluated by a probability measure on $\mathbb{R}_+$. As a particular case, we take the Cesàro average of the running cost over a fixed horizon. The limit of the value with Cesàro average when the horizon tends to infinity is widely studied in the literature. We address the more general question of the existence of a limit for values defined by general evaluations satisfying certain long-term condition.
For this aim, we introduce an asymptotic regularity condition for a sequence of probability measures on $\mathbb{R}_+$. Our main result is that, for any sequence of probability measures on $\mathbb{R}_+$ satisfying this condition, the associated value functions converge uniformly if and only if this family is totally bounded for the uniform norm.
As a byproduct, we obtain the existence of a limit value (for general evaluations) for control systems defined on a compact invariant domain and satisfying suitable nonexpansive property.

Keywords:

Mathematics Subject Classification: 49J15, 93C15, 37A99.

Citation:

Related Papers

Cited by

References

[1]	O. Alvarez and M. Bardi, Ergodicity, stabilization, and singular pertubations for Bellman-Isaacs equations, Mem. Amer. Math. Soc., 204 (2010), vi+77 pp.doi: 10.1090/S0065-9266-09-00588-2.
[2]	M. Arisawa, Ergodic problem for the Hamilton-Jacobi-Belmann equations, Ann. Henri Poincaré, Analyse Nonlinéaire, 14 (1997), 415-438.doi: 10.1016/S0294-1449(97)80134-5.
[3]	M. Arisawa and P. L. Lions, On ergodic stochastic control, Comm. Partial Differential Equations, 23 (1998), 2187-2217.doi: 10.1080/03605309808821413.
[4]	M. Bardi and F. Priuli, LQG Mean-Field Games with ergodic cost, Proc. 52nd IEEE Conference on Decision and Control, (2013), 2493-2498.doi: 10.1109/CDC.2013.6760255.
[5]	A. Bensoussan, Perturbation Methods in Optimal Control, Wiley/Gauthiers-Villas, Chichester, 1988.
[6]	R. Buckdahn, D. Goreac and M. Quincampoix, Existence of asymptotic values for nonexpansive stochastic control systems, Applied Mathematics and Optimization, 70 (2014), 1-28.doi: 10.1007/s00245-013-9230-4.
[7]	V. Gaitsgory, On the use of the averaging method in control problems, (Russian) Differentsialnye Uravneniya, 22 (1986), 1876-1886.
[8]	D. Goreac, A note on general Tauberian-type results for controlled stochastic dynamics, preprint, hal:01120513.
[9]	R. Z. Khasminskii, On the averaging principle for Itô stochastic equations, Kybernetika, 4 (1968), 260-279.
[10]	D. Khlopin, On uniform Tauberian theorems for dynamic games, preprint, arXiv:1412.7331.
[11]	M. Oliu-Barton and G. Vigeral, A uniform Tauberian theorem in optimal control, in Advances in Dynamic Games, Annals of the International Society of Dynamic Games (eds. P. Cardaliaguet and R. Cressman), Birkhauser, 12 (2012), 199-215.
[12]	M. Quincampoix and J. Renault, On the existence of a limit value in some nonexpansive optimal control problems, SIAM Journal on Control and Optimization, 49 (2011), 2118-2132.doi: 10.1137/090756818.
[13]	J. Renault, Uniform value in dynamic programming, J. Eur. Math. Soc. (JEMS), 13 (2011), 309-330.doi: 10.4171/JEMS/254.
[14]	J. Renault, General long-term values in dynamic programming, Journal of Dynamics and Games, 1 (2014), 471-484.
[15]	J. Renault and X. Venel, A distance for probability spaces, and long-term values in Markov decision processes and repeated games, preprint, arXiv:1202.6259.
[16]	S. Sorin, A First Course on Zero-sum Repeated Games, Springer, 2002.
[17]	B. Ziliotto, General limit value in stochastic games, preprint, arXiv:1410.5231.