June  2012, 2(2): 195-215. doi: 10.3934/mcrf.2012.2.195

A unified theory of maximum principle for continuous and discrete time optimal control problems

1. 

Department of Mathematics, Zhejiang University, Zhejiang, Hangzhou, 310027, China, China

2. 

Department of Mathematics, Guizhou University, Guiyang, Guizhou, 550025, China

Received  February 2011 Revised  February 2012 Published  May 2012

Traditionally, the time domains that are widely used in mathematical descriptions are limited to real numbers for the case of continuous-time optimal control problems or to integers for the case of discrete-time optimal control problems. In this paper, based on a family of "needle variations", we derive maximum principle for optimal control problem on time scales. The results not only unify the theory of continuous and discrete optimal control problems but also conclude problems involving time domains in partly continuous and partly discrete ingredients. A simple optimal control problem on time scales is discussed in detail. Meanwhile, the results also unify the theory of some hybrid systems, for example, impulsive systems.
Citation: Zaidong Zhan, Shuping Chen, Wei Wei. A unified theory of maximum principle for continuous and discrete time optimal control problems. Mathematical Control and Related Fields, 2012, 2 (2) : 195-215. doi: 10.3934/mcrf.2012.2.195
References:
[1]

R. Agarwal, V. O. Espinar, K. Perera and D. R. Vivero, Basic properties of Sobolev's space on time scales, Adv. Differential Equations, 2006, Art. ID 38121, 1-14.

[2]

F. M. Atici, D. C. Biles and A. Lebedinsky, An application of time scales to economics, Math. Comput. Modelling., 43 (2006), 718-726. doi: 10.1016/j.mcm.2005.08.014.

[3]

F. M. Atici and F. Uysal, A production-inventory model of HMMS on time scales, Appl. Math. Lett., 21 (2008), 236-243. doi: 10.1016/j.aml.2007.03.013.

[4]

M. Bohner, Calculus of variations on time scales, J. Dynam. Systems Appl., 13 (2004), 339-349.

[5]

M. Bohner and G. S. Gudeinov, Double integral calculus of variations of time scales, Comput. Math. Appl., 54 (2007), 45-57. doi: 10.1016/j.camwa.2006.10.032.

[6]

M. Bohner and A. Peterson, "Dynamic Equations on Time Scales. An Introduction with Applications," Birkhäuser Boston, Inc., Boston, MA, 2001.

[7]

A. G. Butkovskiĭ, The necessary and sufficient optimality conditions for of sampled-data control systems, (Russian), Avtomat. i Telemeh., 24 (1963), 1056-1064.

[8]

A. Cabada and D. R. Vivero, Criterions for absolute continuity on time scales, J. Difference Equ. Appl., 11 (2005), 1013-1028. doi: 10.1080/10236190500272830.

[9]

A. Cabada and D. R. Vivero, Expression of the Lebesgue $\Delta$-integral on time scales as a usual Lebesgue integral: Application to the calculus of $\Delta$-antiderivatives, Math. Comput. Modelling, 43 (2006), 194-207. doi: 10.1016/j.mcm.2005.09.028.

[10]

I. Ekeland, Nonconvex minimization problems, Bull. Amer. Math. Soc. (N.S.), 1 (1979), 443-474.

[11]

Rui A. C. Ferreira and D. F. M. Torres, Higher-order calculus of variations on time scales, in "Mathematical Control Theory and Finance," Springer, Berlin, (2008), 149-159.

[12]

Rui. A. C. Ferreira and D. F. M. Torres, Isoperimetric problems of the calculus of variations on time scales, in "Nonlinear Analysis and Optimization II. Optimization," Contemp. Math., 514, Amer. Math. Soc., Providence, RI, (2010), 123-131.

[13]

J. G. P. Gamarra and R. V. Solé, Complex discrete dynamics from simple continuous population models, Bull. Math. Biol., 64 (2002), 611-620. doi: 10.1006/bulm.2002.0286.

[14]

H. Geering, Continuous-time optimal control theory for cost functionals including discrete state penalty terms, IEEE Trans. Automat. Control., AC-21 (1976), 866-869. doi: 10.1109/TAC.1976.1101377.

[15]

H. Halkin, A maximum principle of the Pontryagin type for systems described by nonlinear difference equations, SIAM J. Control., 4 (1966), 90-111.

[16]

S. Hilger, Analysis on measure chains-a unified approach to continues and discrete calculus, Results Math., 18 (1990), 18-56.

[17]

R. Hilscher and V. Zeidan, Calculus of variations on time scales: Weak local piecewise $C^$$1_{rd}$ solutions with variable endpoints, J. Math. Anal. Appl., 289 (2004), 143-166. doi: 10.1016/j.jmaa.2003.09.031.

[18]

J. M. Holtzman, On the maximum principle for nonlinear discrete-time systems, IEEE Trans. Automat. Control, AC-11 (1966), 273-274. doi: 10.1109/TAC.1966.1098311.

[19]

J. M. Holtzman, Convexity and the maximum principle for discrete systems, IEEE Trans. Automat. Control, AC-11 (1966), 30-35. doi: 10.1109/TAC.1966.1098311.

[20]

F. Horn and R. Jackson, Correspondence discrete maximum principle, Ind. Eng. Chem. Fundamen., 4 (1965), 110-112.

[21]

R. M. May, Simple mathematical models with very complicated dynamics, Nature., 261 (1976), 459-467. doi: 10.1038/261459a0.

[22]

J. A. Oriega and R. J. Leake, Discrete maximum principle with state constrained control, SIAM J. Control. Optim., 15 (1977), 984-990. doi: 10.1137/0315063.

[23]

J. B. Pearson, Jr. and R. Sridhar, A discrete optimal control problem, IEEE Trans. Automat. Control, AC-11 (1966), 171-174. doi: 10.1109/TAC.1966.1098287.

[24]

A. I. Propoĭ, The maximum principle for discrete systems, Autom. Remote Control, 26 (1965), 1167-1177.

[25]

J. B. Rosen, Optimal control and convex programming, in "1967 Nonlinear Programming"(NATO Summer School, Menton, 1964) (ed. J. Abadie), North-Holland, Amsterdam, 287-302.

[26]

L. I. Rozonoèr, L.S. Pontryagin maximum principle of in the theory of optimum systems, Part III, Autom. Remote Control, 20 (1959), 1517-1532.

[27]

C. C. Tisdell and A. Zaidi, Basic qualitative and quantitative results for solutions to nonlinear, dynamic equations on time scales with an application to economic modelling, Nonlinear Anal., 68 (2008), 3504-3524. doi: 10.1016/j.na.2007.03.043.

[28]

J. M. Yong and H. W. Lou, "A Concise Course to Theory of Optimal Control," (in Chinese), Higher Education Press, Beijing, China, 2006.

[29]

Z. Zhan and W. Wei, Necessary conditions for optimal control problems on time scales, Abstr. Appl. Anal., 2009, Art. ID 974394, 14 pp.

[30]

Z. Zhan and W. Wei, On existence of optimal control governed by a class of the first-order linear dynamic systems on time scales, Appl. Math. Comput., 215 (2009), 2070-2081. doi: 10.1016/j.amc.2009.08.009.

[31]

Z. Zhan, W. Wei and Y. F. Li, Optimal control problem with terminal state constraint on time scales, Pacific J. Optim., to appear.

[32]

Z. Zhan, W. Wei and H. Xu, Hamilton-Jacobi-Bellman equations on time scales, Math. Comput. Modelling, 49 (2009), 2019-2028. doi: 10.1016/j.mcm.2008.12.008.

show all references

References:
[1]

R. Agarwal, V. O. Espinar, K. Perera and D. R. Vivero, Basic properties of Sobolev's space on time scales, Adv. Differential Equations, 2006, Art. ID 38121, 1-14.

[2]

F. M. Atici, D. C. Biles and A. Lebedinsky, An application of time scales to economics, Math. Comput. Modelling., 43 (2006), 718-726. doi: 10.1016/j.mcm.2005.08.014.

[3]

F. M. Atici and F. Uysal, A production-inventory model of HMMS on time scales, Appl. Math. Lett., 21 (2008), 236-243. doi: 10.1016/j.aml.2007.03.013.

[4]

M. Bohner, Calculus of variations on time scales, J. Dynam. Systems Appl., 13 (2004), 339-349.

[5]

M. Bohner and G. S. Gudeinov, Double integral calculus of variations of time scales, Comput. Math. Appl., 54 (2007), 45-57. doi: 10.1016/j.camwa.2006.10.032.

[6]

M. Bohner and A. Peterson, "Dynamic Equations on Time Scales. An Introduction with Applications," Birkhäuser Boston, Inc., Boston, MA, 2001.

[7]

A. G. Butkovskiĭ, The necessary and sufficient optimality conditions for of sampled-data control systems, (Russian), Avtomat. i Telemeh., 24 (1963), 1056-1064.

[8]

A. Cabada and D. R. Vivero, Criterions for absolute continuity on time scales, J. Difference Equ. Appl., 11 (2005), 1013-1028. doi: 10.1080/10236190500272830.

[9]

A. Cabada and D. R. Vivero, Expression of the Lebesgue $\Delta$-integral on time scales as a usual Lebesgue integral: Application to the calculus of $\Delta$-antiderivatives, Math. Comput. Modelling, 43 (2006), 194-207. doi: 10.1016/j.mcm.2005.09.028.

[10]

I. Ekeland, Nonconvex minimization problems, Bull. Amer. Math. Soc. (N.S.), 1 (1979), 443-474.

[11]

Rui A. C. Ferreira and D. F. M. Torres, Higher-order calculus of variations on time scales, in "Mathematical Control Theory and Finance," Springer, Berlin, (2008), 149-159.

[12]

Rui. A. C. Ferreira and D. F. M. Torres, Isoperimetric problems of the calculus of variations on time scales, in "Nonlinear Analysis and Optimization II. Optimization," Contemp. Math., 514, Amer. Math. Soc., Providence, RI, (2010), 123-131.

[13]

J. G. P. Gamarra and R. V. Solé, Complex discrete dynamics from simple continuous population models, Bull. Math. Biol., 64 (2002), 611-620. doi: 10.1006/bulm.2002.0286.

[14]

H. Geering, Continuous-time optimal control theory for cost functionals including discrete state penalty terms, IEEE Trans. Automat. Control., AC-21 (1976), 866-869. doi: 10.1109/TAC.1976.1101377.

[15]

H. Halkin, A maximum principle of the Pontryagin type for systems described by nonlinear difference equations, SIAM J. Control., 4 (1966), 90-111.

[16]

S. Hilger, Analysis on measure chains-a unified approach to continues and discrete calculus, Results Math., 18 (1990), 18-56.

[17]

R. Hilscher and V. Zeidan, Calculus of variations on time scales: Weak local piecewise $C^$$1_{rd}$ solutions with variable endpoints, J. Math. Anal. Appl., 289 (2004), 143-166. doi: 10.1016/j.jmaa.2003.09.031.

[18]

J. M. Holtzman, On the maximum principle for nonlinear discrete-time systems, IEEE Trans. Automat. Control, AC-11 (1966), 273-274. doi: 10.1109/TAC.1966.1098311.

[19]

J. M. Holtzman, Convexity and the maximum principle for discrete systems, IEEE Trans. Automat. Control, AC-11 (1966), 30-35. doi: 10.1109/TAC.1966.1098311.

[20]

F. Horn and R. Jackson, Correspondence discrete maximum principle, Ind. Eng. Chem. Fundamen., 4 (1965), 110-112.

[21]

R. M. May, Simple mathematical models with very complicated dynamics, Nature., 261 (1976), 459-467. doi: 10.1038/261459a0.

[22]

J. A. Oriega and R. J. Leake, Discrete maximum principle with state constrained control, SIAM J. Control. Optim., 15 (1977), 984-990. doi: 10.1137/0315063.

[23]

J. B. Pearson, Jr. and R. Sridhar, A discrete optimal control problem, IEEE Trans. Automat. Control, AC-11 (1966), 171-174. doi: 10.1109/TAC.1966.1098287.

[24]

A. I. Propoĭ, The maximum principle for discrete systems, Autom. Remote Control, 26 (1965), 1167-1177.

[25]

J. B. Rosen, Optimal control and convex programming, in "1967 Nonlinear Programming"(NATO Summer School, Menton, 1964) (ed. J. Abadie), North-Holland, Amsterdam, 287-302.

[26]

L. I. Rozonoèr, L.S. Pontryagin maximum principle of in the theory of optimum systems, Part III, Autom. Remote Control, 20 (1959), 1517-1532.

[27]

C. C. Tisdell and A. Zaidi, Basic qualitative and quantitative results for solutions to nonlinear, dynamic equations on time scales with an application to economic modelling, Nonlinear Anal., 68 (2008), 3504-3524. doi: 10.1016/j.na.2007.03.043.

[28]

J. M. Yong and H. W. Lou, "A Concise Course to Theory of Optimal Control," (in Chinese), Higher Education Press, Beijing, China, 2006.

[29]

Z. Zhan and W. Wei, Necessary conditions for optimal control problems on time scales, Abstr. Appl. Anal., 2009, Art. ID 974394, 14 pp.

[30]

Z. Zhan and W. Wei, On existence of optimal control governed by a class of the first-order linear dynamic systems on time scales, Appl. Math. Comput., 215 (2009), 2070-2081. doi: 10.1016/j.amc.2009.08.009.

[31]

Z. Zhan, W. Wei and Y. F. Li, Optimal control problem with terminal state constraint on time scales, Pacific J. Optim., to appear.

[32]

Z. Zhan, W. Wei and H. Xu, Hamilton-Jacobi-Bellman equations on time scales, Math. Comput. Modelling, 49 (2009), 2019-2028. doi: 10.1016/j.mcm.2008.12.008.

[1]

Hans Josef Pesch. Carathéodory's royal road of the calculus of variations: Missed exits to the maximum principle of optimal control theory. Numerical Algebra, Control and Optimization, 2013, 3 (1) : 161-173. doi: 10.3934/naco.2013.3.161

[2]

Zhen Wu, Feng Zhang. Maximum principle for discrete-time stochastic optimal control problem and stochastic game. Mathematical Control and Related Fields, 2022, 12 (2) : 475-493. doi: 10.3934/mcrf.2021031

[3]

Md. Haider Ali Biswas, Maria do Rosário de Pinho. A nonsmooth maximum principle for optimal control problems with state and mixed constraints - convex case. Conference Publications, 2011, 2011 (Special) : 174-183. doi: 10.3934/proc.2011.2011.174

[4]

Stefan Doboszczak, Manil T. Mohan, Sivaguru S. Sritharan. Pontryagin maximum principle for the optimal control of linearized compressible navier-stokes equations with state constraints. Evolution Equations and Control Theory, 2022, 11 (2) : 347-371. doi: 10.3934/eect.2020110

[5]

H. O. Fattorini. The maximum principle for linear infinite dimensional control systems with state constraints. Discrete and Continuous Dynamical Systems, 1995, 1 (1) : 77-101. doi: 10.3934/dcds.1995.1.77

[6]

Shaolin Ji, Xiaole Xue. A stochastic maximum principle for linear quadratic problem with nonconvex control domain. Mathematical Control and Related Fields, 2019, 9 (3) : 495-507. doi: 10.3934/mcrf.2019022

[7]

Gastão S. F. Frederico, Delfim F. M. Torres. Noether's symmetry Theorem for variational and optimal control problems with time delay. Numerical Algebra, Control and Optimization, 2012, 2 (3) : 619-630. doi: 10.3934/naco.2012.2.619

[8]

Huaiqiang Yu, Bin Liu. Pontryagin's principle for local solutions of optimal control governed by the 2D Navier-Stokes equations with mixed control-state constraints. Mathematical Control and Related Fields, 2012, 2 (1) : 61-80. doi: 10.3934/mcrf.2012.2.61

[9]

Torsten Lindström. Discrete models and Fisher's maximum principle in ecology. Conference Publications, 2003, 2003 (Special) : 571-579. doi: 10.3934/proc.2003.2003.571

[10]

Shi'an Wang, N. U. Ahmed. Optimal control and stabilization of building maintenance units based on minimum principle. Journal of Industrial and Management Optimization, 2021, 17 (4) : 1713-1727. doi: 10.3934/jimo.2020041

[11]

Hancheng Guo, Jie Xiong. A second-order stochastic maximum principle for generalized mean-field singular control problem. Mathematical Control and Related Fields, 2018, 8 (2) : 451-473. doi: 10.3934/mcrf.2018018

[12]

Guy Barles, Ariela Briani, Emmanuel Trélat. Value function for regional control problems via dynamic programming and Pontryagin maximum principle. Mathematical Control and Related Fields, 2018, 8 (3&4) : 509-533. doi: 10.3934/mcrf.2018021

[13]

Yunkyong Hyon, Do Young Kwak, Chun Liu. Energetic variational approach in complex fluids: Maximum dissipation principle. Discrete and Continuous Dynamical Systems, 2010, 26 (4) : 1291-1304. doi: 10.3934/dcds.2010.26.1291

[14]

Loïc Bourdin, Emmanuel Trélat. Optimal sampled-data control, and generalizations on time scales. Mathematical Control and Related Fields, 2016, 6 (1) : 53-94. doi: 10.3934/mcrf.2016.6.53

[15]

H. O. Fattorini. The maximum principle in infinite dimension. Discrete and Continuous Dynamical Systems, 2000, 6 (3) : 557-574. doi: 10.3934/dcds.2000.6.557

[16]

Yan Wang, Yanxiang Zhao, Lei Wang, Aimin Song, Yanping Ma. Stochastic maximum principle for partial information optimal investment and dividend problem of an insurer. Journal of Industrial and Management Optimization, 2018, 14 (2) : 653-671. doi: 10.3934/jimo.2017067

[17]

Shanjian Tang. A second-order maximum principle for singular optimal stochastic controls. Discrete and Continuous Dynamical Systems - B, 2010, 14 (4) : 1581-1599. doi: 10.3934/dcdsb.2010.14.1581

[18]

Adel Chala, Dahbia Hafayed. On stochastic maximum principle for risk-sensitive of fully coupled forward-backward stochastic control of mean-field type with application. Evolution Equations and Control Theory, 2020, 9 (3) : 817-843. doi: 10.3934/eect.2020035

[19]

C.Z. Wu, K. L. Teo. Global impulsive optimal control computation. Journal of Industrial and Management Optimization, 2006, 2 (4) : 435-450. doi: 10.3934/jimo.2006.2.435

[20]

Dingjun Yao, Kun Fan. Optimal risk control and dividend strategies in the presence of two reinsurers: Variance premium principle. Journal of Industrial and Management Optimization, 2018, 14 (3) : 1055-1083. doi: 10.3934/jimo.2017090

2021 Impact Factor: 1.141

Metrics

  • PDF downloads (105)
  • HTML views (0)
  • Cited by (5)

Other articles
by authors

[Back to Top]