Limit value for optimal control with general means

Previous Article
Stability analysis of reaction-diffusion models on evolving domains: The effects of cross-diffusion
DCDS Home
This Issue
Next Article
Quartic Julia sets including any two copies of quadratic Julia sets

April 2016, 36(4): 2113-2132. doi: 10.3934/dcds.2016.36.2113

Limit value for optimal control with general means

Xiaoxi Li ^1, , Marc Quincampoix ^2, and Jérôme Renault ^3,

1.	Sorbonne Universités, UPMC Univ Paris 06, IMJ-PRG, UMR 7586, CNRS, Univ Paris Diderot, Sorbonne Paris Cité, 4 Place Jussieu, 75005 Paris, France
2.	Laboratoire de Mathématiques de Bretagne Atlantique, UMR 6205, Université de Brest, 6 Avenue Victor Le Gorgeu, 29200 Brest, France
3.	TSE (GREMAQ, Université Toulouse 1 Capitole and GDR 2932 Théorie des Jeux), 21 allée de Brienne, 31000 Toulouse

Received March 2015 Revised July 2015 Published September 2015

Abstract
Related Papers

We consider optimal control problems where the running cost of the trajectory is evaluated by a probability measure on $\mathbb{R}_+$. As a particular case, we take the Cesàro average of the running cost over a fixed horizon. The limit of the value with Cesàro average when the horizon tends to infinity is widely studied in the literature. We address the more general question of the existence of a limit for values defined by general evaluations satisfying certain long-term condition.
For this aim, we introduce an asymptotic regularity condition for a sequence of probability measures on $\mathbb{R}_+$. Our main result is that, for any sequence of probability measures on $\mathbb{R}_+$ satisfying this condition, the associated value functions converge uniformly if and only if this family is totally bounded for the uniform norm.
As a byproduct, we obtain the existence of a limit value (for general evaluations) for control systems defined on a compact invariant domain and satisfying suitable nonexpansive property.

Keywords: general means., Optimal control, long time average value, limit value.

Mathematics Subject Classification: 49J15, 93C15, 37A9.

Citation: Xiaoxi Li, Marc Quincampoix, Jérôme Renault. Limit value for optimal control with general means. Discrete and Continuous Dynamical Systems, 2016, 36 (4) : 2113-2132. doi: 10.3934/dcds.2016.36.2113

References:

[1]	O. Alvarez and M. Bardi, Ergodicity, stabilization, and singular pertubations for Bellman-Isaacs equations, Mem. Amer. Math. Soc., 204 (2010), vi+77 pp. doi: 10.1090/S0065-9266-09-00588-2.
[2]	M. Arisawa, Ergodic problem for the Hamilton-Jacobi-Belmann equations, Ann. Henri Poincaré, Analyse Nonlinéaire, 14 (1997), 415-438. doi: 10.1016/S0294-1449(97)80134-5.
[3]	M. Arisawa and P. L. Lions, On ergodic stochastic control, Comm. Partial Differential Equations, 23 (1998), 2187-2217. doi: 10.1080/03605309808821413.
[4]	M. Bardi and F. Priuli, LQG Mean-Field Games with ergodic cost, Proc. 52nd IEEE Conference on Decision and Control, (2013), 2493-2498. doi: 10.1109/CDC.2013.6760255.
[5]	A. Bensoussan, Perturbation Methods in Optimal Control, Wiley/Gauthiers-Villas, Chichester, 1988.
[6]	R. Buckdahn, D. Goreac and M. Quincampoix, Existence of asymptotic values for nonexpansive stochastic control systems, Applied Mathematics and Optimization, 70 (2014), 1-28. doi: 10.1007/s00245-013-9230-4.
[7]	V. Gaitsgory, On the use of the averaging method in control problems, (Russian) Differentsialnye Uravneniya, 22 (1986), 1876-1886.
[8]	D. Goreac, A note on general Tauberian-type results for controlled stochastic dynamics, preprint, hal:01120513.
[9]	R. Z. Khasminskii, On the averaging principle for Itô stochastic equations, Kybernetika, 4 (1968), 260-279.
[10]	D. Khlopin, On uniform Tauberian theorems for dynamic games, preprint, arXiv:1412.7331.
[11]	M. Oliu-Barton and G. Vigeral, A uniform Tauberian theorem in optimal control, in Advances in Dynamic Games, Annals of the International Society of Dynamic Games (eds. P. Cardaliaguet and R. Cressman), Birkhauser, 12 (2012), 199-215.
[12]	M. Quincampoix and J. Renault, On the existence of a limit value in some nonexpansive optimal control problems, SIAM Journal on Control and Optimization, 49 (2011), 2118-2132. doi: 10.1137/090756818.
[13]	J. Renault, Uniform value in dynamic programming, J. Eur. Math. Soc. (JEMS), 13 (2011), 309-330. doi: 10.4171/JEMS/254.
[14]	J. Renault, General long-term values in dynamic programming, Journal of Dynamics and Games, 1 (2014), 471-484.
[15]	J. Renault and X. Venel, A distance for probability spaces, and long-term values in Markov decision processes and repeated games, preprint, arXiv:1202.6259.
[16]	S. Sorin, A First Course on Zero-sum Repeated Games, Springer, 2002.
[17]	B. Ziliotto, General limit value in stochastic games, preprint, arXiv:1410.5231.

show all references

References:

[1]	O. Alvarez and M. Bardi, Ergodicity, stabilization, and singular pertubations for Bellman-Isaacs equations, Mem. Amer. Math. Soc., 204 (2010), vi+77 pp. doi: 10.1090/S0065-9266-09-00588-2.
[2]	M. Arisawa, Ergodic problem for the Hamilton-Jacobi-Belmann equations, Ann. Henri Poincaré, Analyse Nonlinéaire, 14 (1997), 415-438. doi: 10.1016/S0294-1449(97)80134-5.
[3]	M. Arisawa and P. L. Lions, On ergodic stochastic control, Comm. Partial Differential Equations, 23 (1998), 2187-2217. doi: 10.1080/03605309808821413.
[4]	M. Bardi and F. Priuli, LQG Mean-Field Games with ergodic cost, Proc. 52nd IEEE Conference on Decision and Control, (2013), 2493-2498. doi: 10.1109/CDC.2013.6760255.
[5]	A. Bensoussan, Perturbation Methods in Optimal Control, Wiley/Gauthiers-Villas, Chichester, 1988.
[6]	R. Buckdahn, D. Goreac and M. Quincampoix, Existence of asymptotic values for nonexpansive stochastic control systems, Applied Mathematics and Optimization, 70 (2014), 1-28. doi: 10.1007/s00245-013-9230-4.
[7]	V. Gaitsgory, On the use of the averaging method in control problems, (Russian) Differentsialnye Uravneniya, 22 (1986), 1876-1886.
[8]	D. Goreac, A note on general Tauberian-type results for controlled stochastic dynamics, preprint, hal:01120513.
[9]	R. Z. Khasminskii, On the averaging principle for Itô stochastic equations, Kybernetika, 4 (1968), 260-279.
[10]	D. Khlopin, On uniform Tauberian theorems for dynamic games, preprint, arXiv:1412.7331.
[11]	M. Oliu-Barton and G. Vigeral, A uniform Tauberian theorem in optimal control, in Advances in Dynamic Games, Annals of the International Society of Dynamic Games (eds. P. Cardaliaguet and R. Cressman), Birkhauser, 12 (2012), 199-215.
[12]	M. Quincampoix and J. Renault, On the existence of a limit value in some nonexpansive optimal control problems, SIAM Journal on Control and Optimization, 49 (2011), 2118-2132. doi: 10.1137/090756818.
[13]	J. Renault, Uniform value in dynamic programming, J. Eur. Math. Soc. (JEMS), 13 (2011), 309-330. doi: 10.4171/JEMS/254.
[14]	J. Renault, General long-term values in dynamic programming, Journal of Dynamics and Games, 1 (2014), 471-484.
[15]	J. Renault and X. Venel, A distance for probability spaces, and long-term values in Markov decision processes and repeated games, preprint, arXiv:1202.6259.
[16]	S. Sorin, A First Course on Zero-sum Repeated Games, Springer, 2002.
[17]	B. Ziliotto, General limit value in stochastic games, preprint, arXiv:1410.5231.

[1]	Jérôme Renault. General limit value in dynamic programming. Journal of Dynamics and Games, 2014, 1 (3) : 471-484. doi: 10.3934/jdg.2014.1.471
[2]	Nguyen Huy Chieu, Jen-Chih Yao. Subgradients of the optimal value function in a parametric discrete optimal control problem. Journal of Industrial and Management Optimization, 2010, 6 (2) : 401-410. doi: 10.3934/jimo.2010.6.401
[3]	Pierre Cardaliaguet, Jean-Michel Lasry, Pierre-Louis Lions, Alessio Porretta. Long time average of mean field games. Networks and Heterogeneous Media, 2012, 7 (2) : 279-301. doi: 10.3934/nhm.2012.7.279
[4]	K. F. Cedric Yiu, S. Y. Wang, K. L. Mak. Optimal portfolios under a value-at-risk constraint with applications to inventory control in supply chains. Journal of Industrial and Management Optimization, 2008, 4 (1) : 81-94. doi: 10.3934/jimo.2008.4.81
[5]	Brahim El Asri. The value of a minimax problem involving impulse control. Journal of Dynamics and Games, 2019, 6 (1) : 1-17. doi: 10.3934/jdg.2019001
[6]	Jingzhen Liu, Lihua Bai, Ka-Fai Cedric Yiu. Optimal investment with a value-at-risk constraint. Journal of Industrial and Management Optimization, 2012, 8 (3) : 531-547. doi: 10.3934/jimo.2012.8.531
[7]	K. Q. Lan, G. C. Yang. Optimal constants for two point boundary value problems. Conference Publications, 2007, 2007 (Special) : 624-633. doi: 10.3934/proc.2007.2007.624
[8]	Andrea Signori. Penalisation of long treatment time and optimal control of a tumour growth model of Cahn–Hilliard type with singular potential. Discrete and Continuous Dynamical Systems, 2021, 41 (6) : 2519-2542. doi: 10.3934/dcds.2020373
[9]	M.J. Lopez-Herrero. The existence of weak solutions for a general class of mixed boundary value problems. Conference Publications, 2011, 2011 (Special) : 1015-1024. doi: 10.3934/proc.2011.2011.1015
[10]	Hui Yang, Yuzhu Han. Initial boundary value problem for a strongly damped wave equation with a general nonlinearity. Evolution Equations and Control Theory, 2022, 11 (3) : 635-648. doi: 10.3934/eect.2021019
[11]	Mark I. Vishik, Sergey Zelik. Attractors for the nonlinear elliptic boundary value problems and their parabolic singular limit. Communications on Pure and Applied Analysis, 2014, 13 (5) : 2059-2093. doi: 10.3934/cpaa.2014.13.2059
[12]	Andrei V. Dmitruk, Nikolai P. Osmolovski. Necessary conditions for a weak minimum in a general optimal control problem with integral equations on a variable time interval. Mathematical Control and Related Fields, 2017, 7 (4) : 507-535. doi: 10.3934/mcrf.2017019
[13]	Ishak Alia. Open-loop equilibriums for a general class of time-inconsistent stochastic optimal control problems. Mathematical Control and Related Fields, 2021 doi: 10.3934/mcrf.2021053
[14]	Yihong Du, Yoshio Yamada. On the long-time limit of positive solutions to the degenerate logistic equation. Discrete and Continuous Dynamical Systems, 2009, 25 (1) : 123-132. doi: 10.3934/dcds.2009.25.123
[15]	Ariela Briani, Hasnaa Zidani. Characterization of the value function of final state constrained control problems with BV trajectories. Communications on Pure and Applied Analysis, 2011, 10 (6) : 1567-1587. doi: 10.3934/cpaa.2011.10.1567
[16]	Gongpin Cheng, Lin Xu. Optimal size of business and dividend strategy in a nonlinear model with refinancing and liquidation value. Mathematical Control and Related Fields, 2017, 7 (1) : 1-19. doi: 10.3934/mcrf.2017001
[17]	Francis Clarke. A general theorem on necessary conditions in optimal control. Discrete and Continuous Dynamical Systems, 2011, 29 (2) : 485-503. doi: 10.3934/dcds.2011.29.485
[18]	Umberto De Maio, Peter I. Kogut, Gabriella Zecca. On optimal $ L^1 $-control in coefficients for quasi-linear Dirichlet boundary value problems with $ BMO $-anisotropic $ p $-Laplacian. Mathematical Control and Related Fields, 2020, 10 (4) : 827-854. doi: 10.3934/mcrf.2020021
[19]	Carlo Sinestrari. Semiconcavity of the value function for exit time problems with nonsmooth target. Communications on Pure and Applied Analysis, 2004, 3 (4) : 757-774. doi: 10.3934/cpaa.2004.3.757
[20]	María Teresa V. Martínez-Palacios, Adrián Hernández-Del-Valle, Ambrosio Ortiz-Ramírez. On the pricing of Asian options with geometric average of American type with stochastic interest rate: A stochastic optimal control approach. Journal of Dynamics and Games, 2019, 6 (1) : 53-64. doi: 10.3934/jdg.2019004

2020 Impact Factor: 1.392

Tools

Metrics

Other articles
by authors

on AIMS
Xiaoxi Li Marc Quincampoix Jérôme Renault
on Google Scholar
Xiaoxi Li Marc Quincampoix Jérôme Renault

[Back to Top]