# American Institute of Mathematical Sciences

2013, 3(1): 109-125. doi: 10.3934/naco.2013.3.109

## An iterative algorithm based on model-reality differences for discrete-time nonlinear stochastic optimal control problems

 1 Department of Mathematics, Universiti Tun Hussein Onn Malaysia, 86400 Parit Raja, Malaysia 2 Department of Mathematics, Universiti Teknologi Malaysia, 81310 UTM, Skudai, Malaysia, Malaysia 3 Department of Mathematics and Statistics, Curtin University, Perth, W.A. 6845

Received  November 2011 Revised  December 2012 Published  January 2013

An iterative algorithm, which is called the integrated optimal control and parameter estimation algorithm, is developed for solving a discrete time nonlinear stochastic control problem. It is based on the integration of the principle of model-reality differences and Kalman filtering theory, where the dynamic integrated system optimization and parameter estimation algorithm are used interactively. In this approach, the weighted least-square output residual is included in the cost function by appropriately monitoring the weighted matrix. An improved linear quadratic Gaussian optimal control model, rather than the original optimal control problem, is solved. Subsequently, the model optimum is updated using the adjusted parameters induced by the differences between the real plant and the model used. These updated solutions converge to the true optimum, despite model-reality differences. For illustration, the optimal control of a nonlinear continuous stirred tank reactor problem is considered and solved by using the method proposed.
Citation: Sie Long Kek, Mohd Ismail Abd Aziz, Kok Lay Teo, Rohanin Ahmad. An iterative algorithm based on model-reality differences for discrete-time nonlinear stochastic optimal control problems. Numerical Algebra, Control and Optimization, 2013, 3 (1) : 109-125. doi: 10.3934/naco.2013.3.109
##### References:
 [1] V. M. Becerra, "Development and Applications of Novel Optimal Control Algorithms," Ph. D. thesis, City University, UK, 1994. [2] V. M. Becerra and P. D. Roberts, Dynamic integrated system optimization and parameter estimation for discrete time optimal control of nonlinear systems, Int. J. Control, 63 (1996), 257-281. doi: 10.1080/00207179608921843. [3] V. M. Becerra and P. D. Roberts, Application of a novel optimal control algorithm to a benchmark fed-batch fermentation process, Trans. Inst. Measurement Control, 20 (1998), 11-18. doi: 10.1177/014233129802000103. [4] A. E. Bryson and Y. C. Ho, "Applied Optimal Control," Hemisphere Publishing Company, New York, 1975. [5] A. E. Bryson, "Applied Linear Optimal Control, Examples and Algorithms," Cambridge University Press, 2002. [6] P. D. Christofides and N. H. El-Farra, "Control of Nonlinear and Hybrid Process Systems: Designs for Uncertainty, Constraints and Time-Delays," Springer-Verlag, New York, 2005. [7] T. E. Dabbous, Adaptive control of nonlinear systems using fussy systems, Journal of Industrial and Management Optimization, 6 (2010), 861-889. doi: 10.3934/jimo.2010.6.861. [8] Y. Y. Haimes and D. A. Wismer, A computational approach to the combined problem of optimization and parameter estimation, Automatica, 8 (1972), 337-347. doi: 10.1016/0005-1098(72)90052-0. [9] M. H. Hu, Q. Gao and H. H. Shao, Optimal control of a class of non-linear discrete-continuous hybrid systems, in "Proceedings of the 6th World Congress on Intelligent Control and Automatic," 21-23 June, 2006, Dalian, China, 835-838. [10] M. H. Hu, Y. S. Wang and H. H Shao, Costate prediction based optimal control for non-linear hybrid systems, ISA Transactions, 47 (2008), 113-118. doi: 10.1016/j.isatra.2007.06.001. [11] S. L. Kek, K. L. Teo and M. I. A. Aziz, An integrated optimal control algorithm for discrete-time nonlinear stochastic system, International Journal of Control, 83 (2010), 2536-2545. doi: 10.1080/00207179.2010.531766. [12] J. S. Kong and B. W. Wan, The study of integrated optimal control approach for complex system under network environment, Computing Technology and Automation, 22 (2003), 23-25. [13] F. L. Lewis, "Optimal Control," John Wiley and Sons, Inc, New York, 1986. [14] F. L. Lewis, "Applied Optimal Control and Estimation: Digital Design and Implementation," Prentice Hall, Inc, 1992. [15] J. M. Li, B. W. Wan and Z. L. Huang, Optimal control of nonlinear discrete systems with model-reality differences, Control Theory and Applications, 16 (1999), 32-37. [16] R. Loxton, K. L. Teo and V. Rehbock, Computational method for a class of switched system optimal control problems, IEEE Transactions on Automatic Control, 54 (2009), 2455-2460. doi: 10.1109/TAC.2009.2029310. [17] R. Loxton, K. L. Teo, V. Rehbock and K. F. C. Yiu, Optimal control problems with a continuous inequality constraint on the state and the control, Automatica, 45 (2009), 2250-2257. doi: 10.1016/j.automatica.2009.05.029. [18] W. H. Ray, "Advanced Process Control," McGraw-Hill, New York, 1989. [19] S. Richchardson and S. Wang, The viscosity approximation to Hamiltonian-Jacobi-Bellman equation in optimal feedback control: upper bounds for extended domains, Journal of Industrial and Management Optimization, 6 (2010), 161-175. doi: 10.3934/jimo.2010.6.161. [20] P. D. Roberts, An algorithm for steady-state system optimization and parameter estimation, Int. J. Systems Science, 10 (1979), 719-734. doi: 10.1080/00207727908941614. [21] P. D. Roberts and T. W. C. Williams, On an algorithm for combined system optimization and parameter estimation, Automatica, 17 (1981), 199-209. doi: 10.1016/0005-1098(81)90095-9. [22] P. D. Roberts, Optimal control of nonlinear systems with model-reality differences, Proceedings of the 31st IEEE Conference on Decision and Control, 1 (1992), 257-258. [23] P. D. Roberts and V. M. Becerra, Optimal control of a class of discrete-continuous non-linear systems decomposition and hierarchical structure, Automatica, 37 (2001), 1757-1769. doi: 10.1016/S0005-1098(01)00141-8. [24] C. Z. Wu, K. L. Teo and V. Rehbock, Optimal control of piecewise affine systems with piecewise affine state feedback, Journal of Industrial and Management Optimization, 5 (2009), 737-747. doi: 10.3934/jimo.2009.5.737. [25] Y. Zhang and S. Y. Li, DISOPE distributed model predictive control of cascade systems with network communication, Journal of Control Theory and Applications, 2 (2005), 131-138. doi: 10.1007/s11768-005-0005-6.

show all references

##### References:
 [1] V. M. Becerra, "Development and Applications of Novel Optimal Control Algorithms," Ph. D. thesis, City University, UK, 1994. [2] V. M. Becerra and P. D. Roberts, Dynamic integrated system optimization and parameter estimation for discrete time optimal control of nonlinear systems, Int. J. Control, 63 (1996), 257-281. doi: 10.1080/00207179608921843. [3] V. M. Becerra and P. D. Roberts, Application of a novel optimal control algorithm to a benchmark fed-batch fermentation process, Trans. Inst. Measurement Control, 20 (1998), 11-18. doi: 10.1177/014233129802000103. [4] A. E. Bryson and Y. C. Ho, "Applied Optimal Control," Hemisphere Publishing Company, New York, 1975. [5] A. E. Bryson, "Applied Linear Optimal Control, Examples and Algorithms," Cambridge University Press, 2002. [6] P. D. Christofides and N. H. El-Farra, "Control of Nonlinear and Hybrid Process Systems: Designs for Uncertainty, Constraints and Time-Delays," Springer-Verlag, New York, 2005. [7] T. E. Dabbous, Adaptive control of nonlinear systems using fussy systems, Journal of Industrial and Management Optimization, 6 (2010), 861-889. doi: 10.3934/jimo.2010.6.861. [8] Y. Y. Haimes and D. A. Wismer, A computational approach to the combined problem of optimization and parameter estimation, Automatica, 8 (1972), 337-347. doi: 10.1016/0005-1098(72)90052-0. [9] M. H. Hu, Q. Gao and H. H. Shao, Optimal control of a class of non-linear discrete-continuous hybrid systems, in "Proceedings of the 6th World Congress on Intelligent Control and Automatic," 21-23 June, 2006, Dalian, China, 835-838. [10] M. H. Hu, Y. S. Wang and H. H Shao, Costate prediction based optimal control for non-linear hybrid systems, ISA Transactions, 47 (2008), 113-118. doi: 10.1016/j.isatra.2007.06.001. [11] S. L. Kek, K. L. Teo and M. I. A. Aziz, An integrated optimal control algorithm for discrete-time nonlinear stochastic system, International Journal of Control, 83 (2010), 2536-2545. doi: 10.1080/00207179.2010.531766. [12] J. S. Kong and B. W. Wan, The study of integrated optimal control approach for complex system under network environment, Computing Technology and Automation, 22 (2003), 23-25. [13] F. L. Lewis, "Optimal Control," John Wiley and Sons, Inc, New York, 1986. [14] F. L. Lewis, "Applied Optimal Control and Estimation: Digital Design and Implementation," Prentice Hall, Inc, 1992. [15] J. M. Li, B. W. Wan and Z. L. Huang, Optimal control of nonlinear discrete systems with model-reality differences, Control Theory and Applications, 16 (1999), 32-37. [16] R. Loxton, K. L. Teo and V. Rehbock, Computational method for a class of switched system optimal control problems, IEEE Transactions on Automatic Control, 54 (2009), 2455-2460. doi: 10.1109/TAC.2009.2029310. [17] R. Loxton, K. L. Teo, V. Rehbock and K. F. C. Yiu, Optimal control problems with a continuous inequality constraint on the state and the control, Automatica, 45 (2009), 2250-2257. doi: 10.1016/j.automatica.2009.05.029. [18] W. H. Ray, "Advanced Process Control," McGraw-Hill, New York, 1989. [19] S. Richchardson and S. Wang, The viscosity approximation to Hamiltonian-Jacobi-Bellman equation in optimal feedback control: upper bounds for extended domains, Journal of Industrial and Management Optimization, 6 (2010), 161-175. doi: 10.3934/jimo.2010.6.161. [20] P. D. Roberts, An algorithm for steady-state system optimization and parameter estimation, Int. J. Systems Science, 10 (1979), 719-734. doi: 10.1080/00207727908941614. [21] P. D. Roberts and T. W. C. Williams, On an algorithm for combined system optimization and parameter estimation, Automatica, 17 (1981), 199-209. doi: 10.1016/0005-1098(81)90095-9. [22] P. D. Roberts, Optimal control of nonlinear systems with model-reality differences, Proceedings of the 31st IEEE Conference on Decision and Control, 1 (1992), 257-258. [23] P. D. Roberts and V. M. Becerra, Optimal control of a class of discrete-continuous non-linear systems decomposition and hierarchical structure, Automatica, 37 (2001), 1757-1769. doi: 10.1016/S0005-1098(01)00141-8. [24] C. Z. Wu, K. L. Teo and V. Rehbock, Optimal control of piecewise affine systems with piecewise affine state feedback, Journal of Industrial and Management Optimization, 5 (2009), 737-747. doi: 10.3934/jimo.2009.5.737. [25] Y. Zhang and S. Y. Li, DISOPE distributed model predictive control of cascade systems with network communication, Journal of Control Theory and Applications, 2 (2005), 131-138. doi: 10.1007/s11768-005-0005-6.
 [1] Sie Long Kek, Kok Lay Teo, Mohd Ismail Abd Aziz. Filtering solution of nonlinear stochastic optimal control problem in discrete-time with model-reality differences. Numerical Algebra, Control and Optimization, 2012, 2 (1) : 207-222. doi: 10.3934/naco.2012.2.207 [2] Sie Long Kek, Mohd Ismail Abd Aziz, Kok Lay Teo. A gradient algorithm for optimal control problems with model-reality differences. Numerical Algebra, Control and Optimization, 2015, 5 (3) : 251-266. doi: 10.3934/naco.2015.5.251 [3] Sie Long Kek, Mohd Ismail Abd Aziz. Output regulation for discrete-time nonlinear stochastic optimal control problems with model-reality differences. Numerical Algebra, Control and Optimization, 2015, 5 (3) : 275-288. doi: 10.3934/naco.2015.5.275 [4] Ellina Grigorieva, Evgenii Khailov. Optimal control of a nonlinear model of economic growth. Conference Publications, 2007, 2007 (Special) : 456-466. doi: 10.3934/proc.2007.2007.456 [5] Kazuyuki Yagasaki. Optimal control of the SIR epidemic model based on dynamical systems theory. Discrete and Continuous Dynamical Systems - B, 2022, 27 (5) : 2501-2513. doi: 10.3934/dcdsb.2021144 [6] Francesco Cordoni, Luca Di Persio. Optimal control for the stochastic FitzHugh-Nagumo model with recovery variable. Evolution Equations and Control Theory, 2018, 7 (4) : 571-585. doi: 10.3934/eect.2018027 [7] Filipe Rodrigues, Cristiana J. Silva, Delfim F. M. Torres, Helmut Maurer. Optimal control of a delayed HIV model. Discrete and Continuous Dynamical Systems - B, 2018, 23 (1) : 443-458. doi: 10.3934/dcdsb.2018030 [8] Chonghu Guan, Fahuai Yi, Xiaoshan Chen. A fully nonlinear free boundary problem arising from optimal dividend and risk control model. Mathematical Control and Related Fields, 2019, 9 (3) : 425-452. doi: 10.3934/mcrf.2019020 [9] Cristiana J. Silva, Helmut Maurer, Delfim F. M. Torres. Optimal control of a Tuberculosis model with state and control delays. Mathematical Biosciences & Engineering, 2017, 14 (1) : 321-337. doi: 10.3934/mbe.2017021 [10] Rong Liu, Feng-Qin Zhang, Yuming Chen. Optimal contraception control for a nonlinear population model with size structure and a separable mortality. Discrete and Continuous Dynamical Systems - B, 2016, 21 (10) : 3603-3618. doi: 10.3934/dcdsb.2016112 [11] Alexandr Golodnikov, Stan Uryasev, Grigoriy Zrazhevsky, Yevgeny Macheret, A. Alexandre Trindade. Optimization of composition and processing parameters for alloy development: a statistical model-based approach. Journal of Industrial and Management Optimization, 2007, 3 (3) : 489-501. doi: 10.3934/jimo.2007.3.489 [12] Rein Luus. Optimal control of oscillatory systems by iterative dynamic programming. Journal of Industrial and Management Optimization, 2008, 4 (1) : 1-15. doi: 10.3934/jimo.2008.4.1 [13] Hang-Chin Lai, Jin-Chirng Lee, Shuh-Jye Chern. A variational problem and optimal control. Journal of Industrial and Management Optimization, 2011, 7 (4) : 967-975. doi: 10.3934/jimo.2011.7.967 [14] Enrique Fernández-Cara, Juan Límaco, Laurent Prouvée. Optimal control of a two-equation model of radiotherapy. Mathematical Control and Related Fields, 2018, 8 (1) : 117-133. doi: 10.3934/mcrf.2018005 [15] Erika Asano, Louis J. Gross, Suzanne Lenhart, Leslie A. Real. Optimal control of vaccine distribution in a rabies metapopulation model. Mathematical Biosciences & Engineering, 2008, 5 (2) : 219-238. doi: 10.3934/mbe.2008.5.219 [16] Huaying Guo, Jin Liang. An optimal control model of carbon reduction and trading. Mathematical Control and Related Fields, 2016, 6 (4) : 535-550. doi: 10.3934/mcrf.2016015 [17] IvÁn Area, FaÏÇal NdaÏrou, Juan J. Nieto, Cristiana J. Silva, Delfim F. M. Torres. Ebola model and optimal control with vaccination constraints. Journal of Industrial and Management Optimization, 2018, 14 (2) : 427-446. doi: 10.3934/jimo.2017054 [18] Qingkai Kong, Zhipeng Qiu, Zi Sang, Yun Zou. Optimal control of a vector-host epidemics model. Mathematical Control and Related Fields, 2011, 1 (4) : 493-508. doi: 10.3934/mcrf.2011.1.493 [19] Erin N. Bodine, Louis J. Gross, Suzanne Lenhart. Optimal control applied to a model for species augmentation. Mathematical Biosciences & Engineering, 2008, 5 (4) : 669-680. doi: 10.3934/mbe.2008.5.669 [20] Hong Niu, Zhijiang Feng, Qijin Xiao, Yajun Zhang. A PID control method based on optimal control strategy. Numerical Algebra, Control and Optimization, 2021, 11 (1) : 117-126. doi: 10.3934/naco.2020019

Impact Factor: