# American Institute of Mathematical Sciences

January  2014, 1(1): 57-78. doi: 10.3934/jdg.2014.1.57

## On the Euler equation approach to discrete--time nonstationary optimal control problems

 1 Departamento de Matemáticas, Instituto Tecnológico Autónomo de México (ITAM), Río Hondo 1, México D.F. 01000, Mexico 2 Mathematics Department, CINVESTAV-IPN, A. Postal 14-740, México D.F. 07000, Mexico

Received  April 2012 Revised  March 2013 Published  June 2013

We are concerned with deterministic and stochastic nonstationary discrete--time optimal control problems in infinite horizon. We show, using Gâteaux differentials, that the so--called Euler equation and a transversality condition are necessary conditions for optimality. In particular, the transversality condition is obtained in a more general form and under milder hypotheses than in previous works. Sufficient conditions are also provided. We also find closed--form solutions to several (discounted) stationary and nonstationary control problems.
Citation: David González-Sánchez, Onésimo Hernández-Lerma. On the Euler equation approach to discrete--time nonstationary optimal control problems. Journal of Dynamics and Games, 2014, 1 (1) : 57-78. doi: 10.3934/jdg.2014.1.57
##### References:
 [1] D. Acemoglu, "Introduction to Modern Economic Growth," Princeton University Press, Princeton, 2009. [2] J. Adda and R. Cooper, "Dynamic Economics. Quantitative Methods and Applications," MIT Press, Cambridge, MA, 2003. [3] V. I. Arkin and I. V. Evstigneev, "Stochastic Models of Control and Economic Dynamics," Academic Press, Orlando, FL, 1987. [4] Y. Bar-Ness, The discrete Euler equation on the normed linear space $l_n^1$, Int. J. Control, 21 (1975), 625-640. doi: 10.1080/00207177508922017. [5] W. A. Brock and L. Mirman, Optimal economic growth and uncertainty: The discounted case, J. Econ. Theory, 4 (1972), 479-513. doi: 10.1016/0022-0531(72)90135-4. [6] J. A. Cadzow, Discrete calculus of variations, Int. J. Control, 11 (1970), 393-407. doi: 10.1080/00207177008905922. [7] G. C. Chow, "Dynamic Economics: Optimization by the Lagrange Method," Oxford University Press, New York, 1997. [8] I. Ekeland and J. A. Scheinkman, Transversality conditions for some infinite horizon discrete time optimization problems, Math. Oper. Res., 11 (1986), 216-229. doi: 10.1287/moor.11.2.216. [9] S. Elaydi, "An Introduction to Difference Equations," Third edition, Undergraduate Texts in Mathematics, Springer, New York, 2005. [10] J. Engwerda, "LQ Dynamic Optimization and Differential Games," John Wiley & Sons, Chichester, 2005. [11] S. Flåm and A. Fougères, Infinite horizon programs; Convergence of approximate solutions, Ann. Oper. Res., 29 (1991), 333-350. doi: 10.1007/BF02283604. [12] W. H. Fleming and R. W. Rishel, "Deterministic and Stochastic Optimal Control," Applications of Mathematics, No. 1, Springer-Verlag, Berlin-New York, 1975. [13] X. Guo, A. Hernández-del-Valle and O. Hernández-Lerma, Nonstationary discrete-time deterministic and stochastic control systems: Bounded and unbounded cases, Systems Control Lett., 60 (2011), 503-509. doi: 10.1016/j.sysconle.2011.04.006. [14] O. Hernández-Lerma and J. B. Lasserre, "Discrete-Time Markov Control Processes: Basic Optimality Criteria," Applications of Mathematics (New York), 30, Springer-Verlag, New York, 1996. [15] T. Kamihigashi, A simple proof of the necessity of the transversality condition, Econ. Theory, 20 (2002), 427-433. doi: 10.1007/s001990100198. [16] T. Kamihigashi, Transversality conditions and dynamic economic behaviour, in "The New Palgrave Dictionary of Economics" (eds. S. N. Durlauf and L. E. Blume), Second edition, Palgrave Macmillan, Hampshire, (2008), 384-387. doi: 10.1057/9780230226203.1737. [17] W. G. Kelley and A. C. Peterson, "Difference Equations. An Introduction with Applications," Academic Press, Inc., Boston, MA, 1991. [18] C. Le Van and R.-A. Dana, "Dynamic Programming in Economics," Dynamic Modeling and Econometrics in Economics and Finance, 5, Kluwer Academic Publishers, Dordrecht, 2003. [19] D. Levhari and L. D. Mirman, The great fish war: An example using dynamic Cournot-Nash solution, Bell J. Econom., 11 (1980), 322-334. doi: 10.2307/3003416. [20] L. Ljungqvist and T. J. Sargent, "Recursive Macroeconomic Theory," Second edition, MIT Press, Cambridge, MA, 2004. [21] D. G. Luenberger, "Optimization by Vector Space Methods," John Wiley & Sons, Inc., New York-London-Sydney, 1969. [22] K. Okuguchi, A dynamic Cournot-Nash equilibrium in fishery: The effects of entry, Riv. Mat. Sci. Econom. Social., 4 (1981), 59-64. doi: 10.1007/BF02123580. [23] W. Rudin, "Principles of Mathematical Analysis," Third edition, McGraw-Hill Book Co., New York-Auckland-Düsseldorf, 1976. [24] I. Schochetman and R. L. Smith, Finite dimensional approximation in infinite-dimensional mathematical programming, Math. Programming, 54 (1992), 307-333. doi: 10.1007/BF01586057. [25] N. L. Stokey, R. E. Lucas and E. C. Prescott, Jr., "Recursive Methods in Economic Dynamics," With the collaboration of Edward C. Prescott, Harvard University Press, Cambridge, MA, 1989. [26] K. Sydsæter, P. J. Hammond, A. Seierstad and A. Strøm, "Further Mathematics for Economic Analysis," Second edition, Prentice-Hall, New York, 2008.

show all references

##### References:
 [1] D. Acemoglu, "Introduction to Modern Economic Growth," Princeton University Press, Princeton, 2009. [2] J. Adda and R. Cooper, "Dynamic Economics. Quantitative Methods and Applications," MIT Press, Cambridge, MA, 2003. [3] V. I. Arkin and I. V. Evstigneev, "Stochastic Models of Control and Economic Dynamics," Academic Press, Orlando, FL, 1987. [4] Y. Bar-Ness, The discrete Euler equation on the normed linear space $l_n^1$, Int. J. Control, 21 (1975), 625-640. doi: 10.1080/00207177508922017. [5] W. A. Brock and L. Mirman, Optimal economic growth and uncertainty: The discounted case, J. Econ. Theory, 4 (1972), 479-513. doi: 10.1016/0022-0531(72)90135-4. [6] J. A. Cadzow, Discrete calculus of variations, Int. J. Control, 11 (1970), 393-407. doi: 10.1080/00207177008905922. [7] G. C. Chow, "Dynamic Economics: Optimization by the Lagrange Method," Oxford University Press, New York, 1997. [8] I. Ekeland and J. A. Scheinkman, Transversality conditions for some infinite horizon discrete time optimization problems, Math. Oper. Res., 11 (1986), 216-229. doi: 10.1287/moor.11.2.216. [9] S. Elaydi, "An Introduction to Difference Equations," Third edition, Undergraduate Texts in Mathematics, Springer, New York, 2005. [10] J. Engwerda, "LQ Dynamic Optimization and Differential Games," John Wiley & Sons, Chichester, 2005. [11] S. Flåm and A. Fougères, Infinite horizon programs; Convergence of approximate solutions, Ann. Oper. Res., 29 (1991), 333-350. doi: 10.1007/BF02283604. [12] W. H. Fleming and R. W. Rishel, "Deterministic and Stochastic Optimal Control," Applications of Mathematics, No. 1, Springer-Verlag, Berlin-New York, 1975. [13] X. Guo, A. Hernández-del-Valle and O. Hernández-Lerma, Nonstationary discrete-time deterministic and stochastic control systems: Bounded and unbounded cases, Systems Control Lett., 60 (2011), 503-509. doi: 10.1016/j.sysconle.2011.04.006. [14] O. Hernández-Lerma and J. B. Lasserre, "Discrete-Time Markov Control Processes: Basic Optimality Criteria," Applications of Mathematics (New York), 30, Springer-Verlag, New York, 1996. [15] T. Kamihigashi, A simple proof of the necessity of the transversality condition, Econ. Theory, 20 (2002), 427-433. doi: 10.1007/s001990100198. [16] T. Kamihigashi, Transversality conditions and dynamic economic behaviour, in "The New Palgrave Dictionary of Economics" (eds. S. N. Durlauf and L. E. Blume), Second edition, Palgrave Macmillan, Hampshire, (2008), 384-387. doi: 10.1057/9780230226203.1737. [17] W. G. Kelley and A. C. Peterson, "Difference Equations. An Introduction with Applications," Academic Press, Inc., Boston, MA, 1991. [18] C. Le Van and R.-A. Dana, "Dynamic Programming in Economics," Dynamic Modeling and Econometrics in Economics and Finance, 5, Kluwer Academic Publishers, Dordrecht, 2003. [19] D. Levhari and L. D. Mirman, The great fish war: An example using dynamic Cournot-Nash solution, Bell J. Econom., 11 (1980), 322-334. doi: 10.2307/3003416. [20] L. Ljungqvist and T. J. Sargent, "Recursive Macroeconomic Theory," Second edition, MIT Press, Cambridge, MA, 2004. [21] D. G. Luenberger, "Optimization by Vector Space Methods," John Wiley & Sons, Inc., New York-London-Sydney, 1969. [22] K. Okuguchi, A dynamic Cournot-Nash equilibrium in fishery: The effects of entry, Riv. Mat. Sci. Econom. Social., 4 (1981), 59-64. doi: 10.1007/BF02123580. [23] W. Rudin, "Principles of Mathematical Analysis," Third edition, McGraw-Hill Book Co., New York-Auckland-Düsseldorf, 1976. [24] I. Schochetman and R. L. Smith, Finite dimensional approximation in infinite-dimensional mathematical programming, Math. Programming, 54 (1992), 307-333. doi: 10.1007/BF01586057. [25] N. L. Stokey, R. E. Lucas and E. C. Prescott, Jr., "Recursive Methods in Economic Dynamics," With the collaboration of Edward C. Prescott, Harvard University Press, Cambridge, MA, 1989. [26] K. Sydsæter, P. J. Hammond, A. Seierstad and A. Strøm, "Further Mathematics for Economic Analysis," Second edition, Prentice-Hall, New York, 2008.
 [1] Le Viet Cuong, Thai Son Doan. Assignability of dichotomy spectra for discrete time-varying linear control systems. Discrete and Continuous Dynamical Systems - B, 2020, 25 (9) : 3597-3607. doi: 10.3934/dcdsb.2020074 [2] Senda Ounaies, Jean-Marc Bonnisseau, Souhail Chebbi, Halil Mete Soner. Merton problem in an infinite horizon and a discrete time with frictions. Journal of Industrial and Management Optimization, 2016, 12 (4) : 1323-1331. doi: 10.3934/jimo.2016.12.1323 [3] Hamid Maarouf. Local Kalman rank condition for linear time varying systems. Mathematical Control and Related Fields, 2022, 12 (2) : 433-446. doi: 10.3934/mcrf.2021029 [4] Vladimir Gaitsgory, Alex Parkinson, Ilya Shvartsman. Linear programming based optimality conditions and approximate solution of a deterministic infinite horizon discounted optimal control problem in discrete time. Discrete and Continuous Dynamical Systems - B, 2019, 24 (4) : 1743-1767. doi: 10.3934/dcdsb.2018235 [5] Vladimir Gaitsgory, Alex Parkinson, Ilya Shvartsman. Linear programming formulations of deterministic infinite horizon optimal control problems in discrete time. Discrete and Continuous Dynamical Systems - B, 2017, 22 (10) : 3821-3838. doi: 10.3934/dcdsb.2017192 [6] Jingang Zhao, Chi Zhang. Finite-horizon optimal control of discrete-time linear systems with completely unknown dynamics using Q-learning. Journal of Industrial and Management Optimization, 2021, 17 (3) : 1471-1483. doi: 10.3934/jimo.2020030 [7] Yi Wang, Dun Zhou. Transversality for time-periodic competitive-cooperative tridiagonal systems. Discrete and Continuous Dynamical Systems - B, 2015, 20 (6) : 1821-1830. doi: 10.3934/dcdsb.2015.20.1821 [8] Xiaoqiang Dai, Chao Yang, Shaobin Huang, Tao Yu, Yuanran Zhu. Finite time blow-up for a wave equation with dynamic boundary condition at critical and high energy levels in control systems. Electronic Research Archive, 2020, 28 (1) : 91-102. doi: 10.3934/era.2020006 [9] Fabio Bagagiolo. An infinite horizon optimal control problem for some switching systems. Discrete and Continuous Dynamical Systems - B, 2001, 1 (4) : 443-462. doi: 10.3934/dcdsb.2001.1.443 [10] Elimhan N. Mahmudov. Second order discrete time-varying and time-invariant linear continuous systems and Kalman type conditions. Numerical Algebra, Control and Optimization, 2022, 12 (2) : 353-371. doi: 10.3934/naco.2021010 [11] Haijun Sun, Xinquan Zhang. Guaranteed cost control of discrete-time switched saturated systems. Discrete and Continuous Dynamical Systems - B, 2021, 26 (8) : 4515-4522. doi: 10.3934/dcdsb.2020300 [12] Abdelfettah Hamzaoui, Nizar Hadj Taieb, Mohamed Ali Hammami. Practical partial stability of time-varying systems. Discrete and Continuous Dynamical Systems - B, 2021  doi: 10.3934/dcdsb.2021197 [13] Hongbiao Fan, Jun-E Feng, Min Meng. Piecewise observers of rectangular discrete fuzzy descriptor systems with multiple time-varying delays. Journal of Industrial and Management Optimization, 2016, 12 (4) : 1535-1556. doi: 10.3934/jimo.2016.12.1535 [14] Tao Pang, Azmat Hussain. An infinite time horizon portfolio optimization model with delays. Mathematical Control and Related Fields, 2016, 6 (4) : 629-651. doi: 10.3934/mcrf.2016018 [15] K. Aruna Sakthi, A. Vinodkumar. Stabilization on input time-varying delay for linear switched systems with truncated predictor control. Numerical Algebra, Control and Optimization, 2020, 10 (2) : 237-247. doi: 10.3934/naco.2019050 [16] Chuandong Li, Fali Ma, Tingwen Huang. 2-D analysis based iterative learning control for linear discrete-time systems with time delay. Journal of Industrial and Management Optimization, 2011, 7 (1) : 175-181. doi: 10.3934/jimo.2011.7.175 [17] Ran Dong, Xuerong Mao. Asymptotic stabilization of continuous-time periodic stochastic systems by feedback control based on periodic discrete-time observations. Mathematical Control and Related Fields, 2020, 10 (4) : 715-734. doi: 10.3934/mcrf.2020017 [18] Valery Y. Glizer, Oleg Kelis. Asymptotic properties of an infinite horizon partial cheap control problem for linear systems with known disturbances. Numerical Algebra, Control and Optimization, 2018, 8 (2) : 211-235. doi: 10.3934/naco.2018013 [19] Tingwen Huang, Guanrong Chen, Juergen Kurths. Synchronization of chaotic systems with time-varying coupling delays. Discrete and Continuous Dynamical Systems - B, 2011, 16 (4) : 1071-1082. doi: 10.3934/dcdsb.2011.16.1071 [20] Karl P. Hadeler. Quiescent phases and stability in discrete time dynamical systems. Discrete and Continuous Dynamical Systems - B, 2015, 20 (1) : 129-152. doi: 10.3934/dcdsb.2015.20.129

Impact Factor: