American Institute of Mathematical Sciences

Advanced
April  2016, 36(4): 2113-2132. doi: 10.3934/dcds.2016.36.2113

Limit value for optimal control with general means

Xiaoxi Li 1, , Marc Quincampoix 2, and  Jérôme Renault 3,

1. 

Sorbonne Universités, UPMC Univ Paris 06, IMJ-PRG, UMR 7586, CNRS, Univ Paris Diderot, Sorbonne Paris Cité, 4 Place Jussieu, 75005 Paris, France
2. 

Laboratoire de Mathématiques de Bretagne Atlantique, UMR 6205, Université de Brest, 6 Avenue Victor Le Gorgeu, 29200 Brest, France
3. 

TSE (GREMAQ, Université Toulouse 1 Capitole and GDR 2932 Théorie des Jeux), 21 allée de Brienne, 31000 Toulouse

Received  March 2015 Revised  July 2015 Published  September 2015

We consider optimal control problems where the running cost of the trajectory is evaluated by a probability measure on $\mathbb{R}_+$. As a particular case, we take the Cesàro average of the running cost over a fixed horizon. The limit of the value with Cesàro average when the horizon tends to infinity is widely studied in the literature. We address the more general question of the existence of a limit for values defined by general evaluations satisfying certain long-term condition.
    For this aim, we introduce an asymptotic regularity condition for a sequence of probability measures on $\mathbb{R}_+$. Our main result is that, for any sequence of probability measures on $\mathbb{R}_+$ satisfying this condition, the associated value functions converge uniformly if and only if this family is totally bounded for the uniform norm.
    As a byproduct, we obtain the existence of a limit value (for general evaluations) for control systems defined on a compact invariant domain and satisfying suitable nonexpansive property.
Keywords: general means., Optimal control, long time average value, limit value.
Mathematics Subject Classification: 49J15, 93C15, 37A9.
Citation: Xiaoxi Li, Marc Quincampoix, Jérôme Renault. Limit value for optimal control with general means. Discrete & Continuous Dynamical Systems, 2016, 36 (4) : 2113-2132. doi: 10.3934/dcds.2016.36.2113
References:
[1]

O. Alvarez and M. Bardi, Ergodicity, stabilization, and singular pertubations for Bellman-Isaacs equations, Mem. Amer. Math. Soc., 204 (2010), vi+77 pp. doi: 10.1090/S0065-9266-09-00588-2.  Google Scholar

[2]

M. Arisawa, Ergodic problem for the Hamilton-Jacobi-Belmann equations, Ann. Henri Poincaré, Analyse Nonlinéaire, 14 (1997), 415-438. doi: 10.1016/S0294-1449(97)80134-5.  Google Scholar

[3]

M. Arisawa and P. L. Lions, On ergodic stochastic control, Comm. Partial Differential Equations, 23 (1998), 2187-2217. doi: 10.1080/03605309808821413.  Google Scholar

[4]

M. Bardi and F. Priuli, LQG Mean-Field Games with ergodic cost, Proc. 52nd IEEE Conference on Decision and Control, (2013), 2493-2498. doi: 10.1109/CDC.2013.6760255.  Google Scholar

[5]

A. Bensoussan, Perturbation Methods in Optimal Control, Wiley/Gauthiers-Villas, Chichester, 1988.  Google Scholar

[6]

R. Buckdahn, D. Goreac and M. Quincampoix, Existence of asymptotic values for nonexpansive stochastic control systems, Applied Mathematics and Optimization, 70 (2014), 1-28. doi: 10.1007/s00245-013-9230-4.  Google Scholar

[7]

V. Gaitsgory, On the use of the averaging method in control problems, (Russian) Differentsialnye Uravneniya, 22 (1986), 1876-1886.  Google Scholar

[8]

D. Goreac, A note on general Tauberian-type results for controlled stochastic dynamics,, preprint, ().   Google Scholar

[9]

R. Z. Khasminskii, On the averaging principle for Itô stochastic equations, Kybernetika, 4 (1968), 260-279.  Google Scholar

[10]

D. Khlopin, On uniform Tauberian theorems for dynamic games,, preprint, ().   Google Scholar

[11]

M. Oliu-Barton and G. Vigeral, A uniform Tauberian theorem in optimal control, in Advances in Dynamic Games, Annals of the International Society of Dynamic Games (eds. P. Cardaliaguet and R. Cressman), Birkhauser, 12 (2012), 199-215.  Google Scholar

[12]

M. Quincampoix and J. Renault, On the existence of a limit value in some nonexpansive optimal control problems, SIAM Journal on Control and Optimization, 49 (2011), 2118-2132. doi: 10.1137/090756818.  Google Scholar

[13]

J. Renault, Uniform value in dynamic programming, J. Eur. Math. Soc. (JEMS), 13 (2011), 309-330. doi: 10.4171/JEMS/254.  Google Scholar

[14]

J. Renault, General long-term values in dynamic programming, Journal of Dynamics and Games, 1 (2014), 471-484.  Google Scholar

[15]

J. Renault and X. Venel, A distance for probability spaces, and long-term values in Markov decision processes and repeated games,, preprint, ().   Google Scholar

[16]

S. Sorin, A First Course on Zero-sum Repeated Games, Springer, 2002.  Google Scholar

[17]

B. Ziliotto, General limit value in stochastic games,, preprint, ().   Google Scholar

show all references

References:
[1]

O. Alvarez and M. Bardi, Ergodicity, stabilization, and singular pertubations for Bellman-Isaacs equations, Mem. Amer. Math. Soc., 204 (2010), vi+77 pp. doi: 10.1090/S0065-9266-09-00588-2.  Google Scholar

[2]

M. Arisawa, Ergodic problem for the Hamilton-Jacobi-Belmann equations, Ann. Henri Poincaré, Analyse Nonlinéaire, 14 (1997), 415-438. doi: 10.1016/S0294-1449(97)80134-5.  Google Scholar

[3]

M. Arisawa and P. L. Lions, On ergodic stochastic control, Comm. Partial Differential Equations, 23 (1998), 2187-2217. doi: 10.1080/03605309808821413.  Google Scholar

[4]

M. Bardi and F. Priuli, LQG Mean-Field Games with ergodic cost, Proc. 52nd IEEE Conference on Decision and Control, (2013), 2493-2498. doi: 10.1109/CDC.2013.6760255.  Google Scholar

[5]

A. Bensoussan, Perturbation Methods in Optimal Control, Wiley/Gauthiers-Villas, Chichester, 1988.  Google Scholar

[6]

R. Buckdahn, D. Goreac and M. Quincampoix, Existence of asymptotic values for nonexpansive stochastic control systems, Applied Mathematics and Optimization, 70 (2014), 1-28. doi: 10.1007/s00245-013-9230-4.  Google Scholar

[7]

V. Gaitsgory, On the use of the averaging method in control problems, (Russian) Differentsialnye Uravneniya, 22 (1986), 1876-1886.  Google Scholar

[8]

D. Goreac, A note on general Tauberian-type results for controlled stochastic dynamics,, preprint, ().   Google Scholar

[9]

R. Z. Khasminskii, On the averaging principle for Itô stochastic equations, Kybernetika, 4 (1968), 260-279.  Google Scholar

[10]

D. Khlopin, On uniform Tauberian theorems for dynamic games,, preprint, ().   Google Scholar

[11]

M. Oliu-Barton and G. Vigeral, A uniform Tauberian theorem in optimal control, in Advances in Dynamic Games, Annals of the International Society of Dynamic Games (eds. P. Cardaliaguet and R. Cressman), Birkhauser, 12 (2012), 199-215.  Google Scholar

[12]

M. Quincampoix and J. Renault, On the existence of a limit value in some nonexpansive optimal control problems, SIAM Journal on Control and Optimization, 49 (2011), 2118-2132. doi: 10.1137/090756818.  Google Scholar

[13]

J. Renault, Uniform value in dynamic programming, J. Eur. Math. Soc. (JEMS), 13 (2011), 309-330. doi: 10.4171/JEMS/254.  Google Scholar

[14]

J. Renault, General long-term values in dynamic programming, Journal of Dynamics and Games, 1 (2014), 471-484.  Google Scholar

[15]

J. Renault and X. Venel, A distance for probability spaces, and long-term values in Markov decision processes and repeated games,, preprint, ().   Google Scholar

[16]

S. Sorin, A First Course on Zero-sum Repeated Games, Springer, 2002.  Google Scholar

[17]

B. Ziliotto, General limit value in stochastic games,, preprint, ().   Google Scholar

[1]

Jérôme Renault. General limit value in dynamic programming. Journal of Dynamics & Games, 2014, 1 (3) : 471-484. doi: 10.3934/jdg.2014.1.471
[2]

Nguyen Huy Chieu, Jen-Chih Yao. Subgradients of the optimal value function in a parametric discrete optimal control problem. Journal of Industrial & Management Optimization, 2010, 6 (2) : 401-410. doi: 10.3934/jimo.2010.6.401
[3]

Pierre Cardaliaguet, Jean-Michel Lasry, Pierre-Louis Lions, Alessio Porretta. Long time average of mean field games. Networks & Heterogeneous Media, 2012, 7 (2) : 279-301. doi: 10.3934/nhm.2012.7.279
[4]

K. F. Cedric Yiu, S. Y. Wang, K. L. Mak. Optimal portfolios under a value-at-risk constraint with applications to inventory control in supply chains. Journal of Industrial & Management Optimization, 2008, 4 (1) : 81-94. doi: 10.3934/jimo.2008.4.81
[5]

Brahim El Asri. The value of a minimax problem involving impulse control. Journal of Dynamics & Games, 2019, 6 (1) : 1-17. doi: 10.3934/jdg.2019001
[6]

Jingzhen Liu, Lihua Bai, Ka-Fai Cedric Yiu. Optimal investment with a value-at-risk constraint. Journal of Industrial & Management Optimization, 2012, 8 (3) : 531-547. doi: 10.3934/jimo.2012.8.531
[7]

K. Q. Lan, G. C. Yang. Optimal constants for two point boundary value problems. Conference Publications, 2007, 2007 (Special) : 624-633. doi: 10.3934/proc.2007.2007.624
[8]

Andrea Signori. Penalisation of long treatment time and optimal control of a tumour growth model of Cahn–Hilliard type with singular potential. Discrete & Continuous Dynamical Systems, 2021, 41 (6) : 2519-2542. doi: 10.3934/dcds.2020373
[9]

M.J. Lopez-Herrero. The existence of weak solutions for a general class of mixed boundary value problems. Conference Publications, 2011, 2011 (Special) : 1015-1024. doi: 10.3934/proc.2011.2011.1015
[10]

Hui Yang, Yuzhu Han. Initial boundary value problem for a strongly damped wave equation with a general nonlinearity. Evolution Equations & Control Theory, 2021  doi: 10.3934/eect.2021019
[11]

Mark I. Vishik, Sergey Zelik. Attractors for the nonlinear elliptic boundary value problems and their parabolic singular limit. Communications on Pure & Applied Analysis, 2014, 13 (5) : 2059-2093. doi: 10.3934/cpaa.2014.13.2059
[12]

Andrei V. Dmitruk, Nikolai P. Osmolovski. Necessary conditions for a weak minimum in a general optimal control problem with integral equations on a variable time interval. Mathematical Control & Related Fields, 2017, 7 (4) : 507-535. doi: 10.3934/mcrf.2017019
[13]

Ishak Alia. Open-loop equilibriums for a general class of time-inconsistent stochastic optimal control problems. Mathematical Control & Related Fields, 2021  doi: 10.3934/mcrf.2021053
[14]

Yihong Du, Yoshio Yamada. On the long-time limit of positive solutions to the degenerate logistic equation. Discrete & Continuous Dynamical Systems, 2009, 25 (1) : 123-132. doi: 10.3934/dcds.2009.25.123
[15]

Ariela Briani, Hasnaa Zidani. Characterization of the value function of final state constrained control problems with BV trajectories. Communications on Pure & Applied Analysis, 2011, 10 (6) : 1567-1587. doi: 10.3934/cpaa.2011.10.1567
[16]

Gongpin Cheng, Lin Xu. Optimal size of business and dividend strategy in a nonlinear model with refinancing and liquidation value. Mathematical Control & Related Fields, 2017, 7 (1) : 1-19. doi: 10.3934/mcrf.2017001
[17]

Francis Clarke. A general theorem on necessary conditions in optimal control. Discrete & Continuous Dynamical Systems, 2011, 29 (2) : 485-503. doi: 10.3934/dcds.2011.29.485
[18]

Umberto De Maio, Peter I. Kogut, Gabriella Zecca. On optimal $ L^1 $-control in coefficients for quasi-linear Dirichlet boundary value problems with $ BMO $-anisotropic $ p $-Laplacian. Mathematical Control & Related Fields, 2020, 10 (4) : 827-854. doi: 10.3934/mcrf.2020021
[19]

Carlo Sinestrari. Semiconcavity of the value function for exit time problems with nonsmooth target. Communications on Pure & Applied Analysis, 2004, 3 (4) : 757-774. doi: 10.3934/cpaa.2004.3.757
[20]

María Teresa V. Martínez-Palacios, Adrián Hernández-Del-Valle, Ambrosio Ortiz-Ramírez. On the pricing of Asian options with geometric average of American type with stochastic interest rate: A stochastic optimal control approach. Journal of Dynamics & Games, 2019, 6 (1) : 53-64. doi: 10.3934/jdg.2019004

2020 Impact Factor: 1.392

Tools

Metrics

  • PDF downloads (106)
  • HTML views (0)
  • Cited by (6)

Other articles
by authors

[Back to Top]

Copyright © 2022 American Institute of Mathematical Sciences | Privacy Policy