
-
Previous Article
Piecewise quadratic bounding functions for finding real roots of polynomials
- NACO Home
- This Issue
-
Next Article
On the GSOR iteration method for image restoration
On the bang-bang control approach via a component-wise line search strategy for unconstrained optimization
1. | School of Mathematical Sciences, Sunway University, Selangor, Malaysia |
2. | Department of Aerospace and Software Engineering, Gyeongsang National University, Jinju, Republic of Korea |
3. | School of Electrical Engineering, Computing and Mathematical Sciences, Curtin University, Perth, Australia |
4. | Department of Electrical and Computer Engineering, Curtin University, Sarawak, Malaysia |
A bang-bang iteration method equipped with a component-wise line search strategy is introduced to solve unconstrained optimization problems. The main idea of this method is to formulate an unconstrained optimization problem as an optimal control problem to obtain an optimal trajectory. However, the optimal trajectory can only be generated by impulsive control variables and it is a straight line joining a guessed initial point to a minimum point. Thus, a priori bounds are imposed on the control variables in order to obtain a feasible solution. As a result, the optimal trajectory is made up of bang-bang control sub-arcs, which form an iterative model based on the Lyapunov function's theorem. This is to ensure monotonic decrease of the objective function value and convergence to a desirable minimum point. However, a chattering behavior may occur near the solution. To avoid this behavior, the Newton iterations are then applied to the proposed method via a two-phase approach to achieve fast convergence. Numerical experiments show that this new approach is efficient and cost-effective to solve the unconstrained optimization problems.
References:
[1] |
N. Andrei,
An unconstrained optimization test functions collection, Adv. Model. Optim., 10 (2008), 147-161.
|
[2] |
D. Bushaw, Optimal discontinuous forcing terms, in Contributions to the Theory of Nonlinear Oscillations, Vol. IV (ed. S. Lefshetz), Princeton Univ. Press, Princeton, New Jersey, (1958), 29–52. |
[3] |
E. D. Dolan and J. J. Moré,
Benchmarking optimization software with performance profiles, Mathematical Programming, 91 (2002), 201-213.
doi: 10.1007/s101070100263. |
[4] |
R. Fletcher, Practical Methods of Optimization Volume 1: Unconstrained Optimization, John Wiley & Sons, 1987. |
[5] |
I. Flüotz and H. Junr, Optimum and quasi-optimum control of third-and fourth-order systems, Technical Report 2, 1963. |
[6] |
A. Fuller, Relay control systems optimized for various performance criteria, in Automatic and Remote Control, Proc. First World Congress IFAC Moscow, vol. 1, 1960, 510–519. |
[7] |
P. E. Gill, W. Murray, M. A. Saunders and M. H. Wright,
Computing forward-difference intervals for numerical optimization, SIAM Journal on Scientific and Statistical Computing, 4 (1983), 310-321.
doi: 10.1137/0904025. |
[8] |
B. S. Goh,
Algorithms for unconstrained optimization problems via control theory, Journal of Optimization Theory and Applications, 92 (1997), 581-604.
doi: 10.1023/A:1022607507153. |
[9] |
B. S. Goh,
Greatest descent algorithms in unconstrained optimization, Journal of Optimization Theory and Applications, 142 (2009), 275-289.
doi: 10.1007/s10957-009-9533-4. |
[10] |
B. S. Goh,
Convergence of algorithms in optimization and solutions of nonlinear equations, Journal of Optimization Theory and Applications, 144 (2010), 43-55.
doi: 10.1007/s10957-009-9583-7. |
[11] |
B. S. Goh,
Approximate greatest descent methods for optimization with equality constraints, Journal of Optimization Theory and Applications, 148 (2011), 505-527.
doi: 10.1007/s10957-010-9765-3. |
[12] |
B. S. Goh, W. J. Leong and K. L. Teo, Robustness of convergence proofs in numerical methods in unconstrained optimization, in Optimization and Control Methods in Industrial Engineering and Construction, Intelligent Systems, Control and Automation: Science and Engineering 72, Springer Netherlands, (2014), 1–9. |
[13] |
B. S. Goh and D. B. McDonald,
Newton methods to solve a system of nonlinear algebraic equations, Journal of Optimization Theory and Applications, 164 (2015), 261-276.
doi: 10.1007/s10957-014-0544-4. |
[14] |
A. Hedar, Studies on Metaheuristics for Continuous Global Optimization Problems, PhD Thesis, Kyoto University, Japan, 2004. |
[15] |
J. Kowalik and J. Morrison,
Analysis of kinetic data for allosteric enzyme reactions as a nonlinear regression problem, Mathematical Biosciences, 2 (1968), 57-66.
|
[16] |
I. Kupka,
The ubiquity of fuller's phenomenon, Nonlinear Controllability and Optimal Control, 133 (1990), 313-350.
|
[17] |
J. P. LaSalle, Time optimal control systems, Proceedings of the National Academy of Sciences of the United States of America.
doi: 10.1073/pnas.45.4.573. |
[18] |
J. P. LaSalle,
Recent advances in liapunov stability theory, SIAM Review, 6 (1964), 1-11.
doi: 10.1137/1006001. |
[19] |
M. S. Lee, B. S. Goh, H. G. Harno and K. H. Lim,
On a two-phase approximate greatest descent method for nonlinear optimization with equality constraints, Numerical Algebra, Control & Optimization, 8 (2018), 325-336.
doi: 10.3934/naco.2018020. |
[20] |
G. Leitmann, The Calculus of Variations and Optimal Control, Vol. 20 of Mathematical Concepts and Methods in Science and Engineering, Plenum Press, New York, 1981.
![]() ![]() |
[21] |
J. J. Moré, B. S. Garbow and K. E. Hillstrom,
Testing unconstrained optimization software, ACM Transactions on Mathematical Software (TOMS), 7 (1981), 17-41.
doi: 10.1145/355934.355936. |
[22] |
J. M. Ortega,
Stability of difference equations and convergence of iterative processes, SIAM Journal on Numerical Analysis, 10 (1973), 268-282.
doi: 10.1137/0710026. |
[23] |
J. M. Ortega and W. C. Rheinboldt, Iterative Solution of Nonlinear Equations in Several Variables, Academic Press, NY, 1970.
![]() ![]() |
[24] |
B. T. Polyak,
Newton's method and its use in optimization, European Journal of Operational Research, 181 (2007), 1086-1096.
doi: 10.1016/j.ejor.2005.06.076. |
[25] |
M. I. Zelikin and V. F. Borisov,
Regimes with increasingly more frequent switchings in optimal control problems, Tr. Mat. Inst. Akad. Nauk SSSR, 197 (1991), 85-167.
|
[26] |
M. I. Zelikin and V. F. Borisov, Theory of Chattering Control: With Applications to Astronautics, Robotics, Economics, and Engineering, Springer Science & Business Media, 2012.
doi: 10.1007/978-1-4612-2702-1. |
show all references
References:
[1] |
N. Andrei,
An unconstrained optimization test functions collection, Adv. Model. Optim., 10 (2008), 147-161.
|
[2] |
D. Bushaw, Optimal discontinuous forcing terms, in Contributions to the Theory of Nonlinear Oscillations, Vol. IV (ed. S. Lefshetz), Princeton Univ. Press, Princeton, New Jersey, (1958), 29–52. |
[3] |
E. D. Dolan and J. J. Moré,
Benchmarking optimization software with performance profiles, Mathematical Programming, 91 (2002), 201-213.
doi: 10.1007/s101070100263. |
[4] |
R. Fletcher, Practical Methods of Optimization Volume 1: Unconstrained Optimization, John Wiley & Sons, 1987. |
[5] |
I. Flüotz and H. Junr, Optimum and quasi-optimum control of third-and fourth-order systems, Technical Report 2, 1963. |
[6] |
A. Fuller, Relay control systems optimized for various performance criteria, in Automatic and Remote Control, Proc. First World Congress IFAC Moscow, vol. 1, 1960, 510–519. |
[7] |
P. E. Gill, W. Murray, M. A. Saunders and M. H. Wright,
Computing forward-difference intervals for numerical optimization, SIAM Journal on Scientific and Statistical Computing, 4 (1983), 310-321.
doi: 10.1137/0904025. |
[8] |
B. S. Goh,
Algorithms for unconstrained optimization problems via control theory, Journal of Optimization Theory and Applications, 92 (1997), 581-604.
doi: 10.1023/A:1022607507153. |
[9] |
B. S. Goh,
Greatest descent algorithms in unconstrained optimization, Journal of Optimization Theory and Applications, 142 (2009), 275-289.
doi: 10.1007/s10957-009-9533-4. |
[10] |
B. S. Goh,
Convergence of algorithms in optimization and solutions of nonlinear equations, Journal of Optimization Theory and Applications, 144 (2010), 43-55.
doi: 10.1007/s10957-009-9583-7. |
[11] |
B. S. Goh,
Approximate greatest descent methods for optimization with equality constraints, Journal of Optimization Theory and Applications, 148 (2011), 505-527.
doi: 10.1007/s10957-010-9765-3. |
[12] |
B. S. Goh, W. J. Leong and K. L. Teo, Robustness of convergence proofs in numerical methods in unconstrained optimization, in Optimization and Control Methods in Industrial Engineering and Construction, Intelligent Systems, Control and Automation: Science and Engineering 72, Springer Netherlands, (2014), 1–9. |
[13] |
B. S. Goh and D. B. McDonald,
Newton methods to solve a system of nonlinear algebraic equations, Journal of Optimization Theory and Applications, 164 (2015), 261-276.
doi: 10.1007/s10957-014-0544-4. |
[14] |
A. Hedar, Studies on Metaheuristics for Continuous Global Optimization Problems, PhD Thesis, Kyoto University, Japan, 2004. |
[15] |
J. Kowalik and J. Morrison,
Analysis of kinetic data for allosteric enzyme reactions as a nonlinear regression problem, Mathematical Biosciences, 2 (1968), 57-66.
|
[16] |
I. Kupka,
The ubiquity of fuller's phenomenon, Nonlinear Controllability and Optimal Control, 133 (1990), 313-350.
|
[17] |
J. P. LaSalle, Time optimal control systems, Proceedings of the National Academy of Sciences of the United States of America.
doi: 10.1073/pnas.45.4.573. |
[18] |
J. P. LaSalle,
Recent advances in liapunov stability theory, SIAM Review, 6 (1964), 1-11.
doi: 10.1137/1006001. |
[19] |
M. S. Lee, B. S. Goh, H. G. Harno and K. H. Lim,
On a two-phase approximate greatest descent method for nonlinear optimization with equality constraints, Numerical Algebra, Control & Optimization, 8 (2018), 325-336.
doi: 10.3934/naco.2018020. |
[20] |
G. Leitmann, The Calculus of Variations and Optimal Control, Vol. 20 of Mathematical Concepts and Methods in Science and Engineering, Plenum Press, New York, 1981.
![]() ![]() |
[21] |
J. J. Moré, B. S. Garbow and K. E. Hillstrom,
Testing unconstrained optimization software, ACM Transactions on Mathematical Software (TOMS), 7 (1981), 17-41.
doi: 10.1145/355934.355936. |
[22] |
J. M. Ortega,
Stability of difference equations and convergence of iterative processes, SIAM Journal on Numerical Analysis, 10 (1973), 268-282.
doi: 10.1137/0710026. |
[23] |
J. M. Ortega and W. C. Rheinboldt, Iterative Solution of Nonlinear Equations in Several Variables, Academic Press, NY, 1970.
![]() ![]() |
[24] |
B. T. Polyak,
Newton's method and its use in optimization, European Journal of Operational Research, 181 (2007), 1086-1096.
doi: 10.1016/j.ejor.2005.06.076. |
[25] |
M. I. Zelikin and V. F. Borisov,
Regimes with increasingly more frequent switchings in optimal control problems, Tr. Mat. Inst. Akad. Nauk SSSR, 197 (1991), 85-167.
|
[26] |
M. I. Zelikin and V. F. Borisov, Theory of Chattering Control: With Applications to Astronautics, Robotics, Economics, and Engineering, Springer Science & Business Media, 2012.
doi: 10.1007/978-1-4612-2702-1. |




Phase-Ⅰ | |||
0 | |||
1 | |||
2 | |||
3 | |||
4 | |||
5 | |||
6 | |||
7 | |||
8 | |||
9 | |||
10 | |||
11 | |||
12 | |||
13 | |||
14 | |||
Phase-Ⅱ | |||
{det H} | |||
0 | |||
1 |
Phase-Ⅰ | |||
0 | |||
1 | |||
2 | |||
3 | |||
4 | |||
5 | |||
6 | |||
7 | |||
8 | |||
9 | |||
10 | |||
11 | |||
12 | |||
13 | |||
14 | |||
Phase-Ⅱ | |||
{det H} | |||
0 | |||
1 |
Phase-Ⅰ | |||
0 | |||
1 | |||
2 | |||
3 | |||
Phase-Ⅱ | |||
{det H} | |||
0 | |||
1 | |||
2 | |||
3 | |||
4 |
Phase-Ⅰ | |||
0 | |||
1 | |||
2 | |||
3 | |||
Phase-Ⅱ | |||
{det H} | |||
0 | |||
1 | |||
2 | |||
3 | |||
4 |
Methods | k | CPU Time (s) | ||
AGDS | 37 | |||
BTR | 81 | |||
AGDRN | 500 | |||
AGD-RS | 83 |
Methods | k | CPU Time (s) | ||
AGDS | 37 | |||
BTR | 81 | |||
AGDRN | 500 | |||
AGD-RS | 83 |
Methods | CPU Time (s) | |||
AGDS | 14 | |||
BTR | 16 | |||
AGDRN | 499 | |||
AGD-RS | 14 |
Methods | CPU Time (s) | |||
AGDS | 14 | |||
BTR | 16 | |||
AGDRN | 499 | |||
AGD-RS | 14 |
[1] |
M. S. Lee, B. S. Goh, H. G. Harno, K. H. Lim. On a two-phase approximate greatest descent method for nonlinear optimization with equality constraints. Numerical Algebra, Control and Optimization, 2018, 8 (3) : 315-326. doi: 10.3934/naco.2018020 |
[2] |
Gengsheng Wang, Yubiao Zhang. Decompositions and bang-bang properties. Mathematical Control and Related Fields, 2017, 7 (1) : 73-170. doi: 10.3934/mcrf.2017005 |
[3] |
King Hann Lim, Hong Hui Tan, Hendra G. Harno. Approximate greatest descent in neural network optimization. Numerical Algebra, Control and Optimization, 2018, 8 (3) : 327-336. doi: 10.3934/naco.2018021 |
[4] |
Helmut Maurer, Tanya Tarnopolskaya, Neale Fulton. Computation of bang-bang and singular controls in collision avoidance. Journal of Industrial and Management Optimization, 2014, 10 (2) : 443-460. doi: 10.3934/jimo.2014.10.443 |
[5] |
Karl Kunisch, Lijuan Wang. The bang-bang property of time optimal controls for the Burgers equation. Discrete and Continuous Dynamical Systems, 2014, 34 (9) : 3611-3637. doi: 10.3934/dcds.2014.34.3611 |
[6] |
Karl Kunisch, Lijuan Wang. Bang-bang property of time optimal controls of semilinear parabolic equation. Discrete and Continuous Dynamical Systems, 2016, 36 (1) : 279-302. doi: 10.3934/dcds.2016.36.279 |
[7] |
Jianjun Liu, Min Zeng, Yifan Ge, Changzhi Wu, Xiangyu Wang. Improved Cuckoo Search algorithm for numerical function optimization. Journal of Industrial and Management Optimization, 2020, 16 (1) : 103-115. doi: 10.3934/jimo.2018142 |
[8] |
Walter Alt, Robert Baier, Matthias Gerdts, Frank Lempio. Error bounds for Euler approximation of linear-quadratic control problems with bang-bang solutions. Numerical Algebra, Control and Optimization, 2012, 2 (3) : 547-570. doi: 10.3934/naco.2012.2.547 |
[9] |
Zhong Wan, Chaoming Hu, Zhanlu Yang. A spectral PRP conjugate gradient methods for nonconvex optimization problem based on modified line search. Discrete and Continuous Dynamical Systems - B, 2011, 16 (4) : 1157-1169. doi: 10.3934/dcdsb.2011.16.1157 |
[10] |
Wataru Nakamura, Yasushi Narushima, Hiroshi Yabe. Nonlinear conjugate gradient methods with sufficient descent properties for unconstrained optimization. Journal of Industrial and Management Optimization, 2013, 9 (3) : 595-619. doi: 10.3934/jimo.2013.9.595 |
[11] |
J. Leonel Rocha, Abdel-Kaddous Taha, Danièle Fournier-Prunaret. Explosion birth and extinction: Double big bang bifurcations and Allee effect in Tsoularis-Wallace's growth models. Discrete and Continuous Dynamical Systems - B, 2015, 20 (9) : 3131-3163. doi: 10.3934/dcdsb.2015.20.3131 |
[12] |
Dieter Bothe, Jan Prüss. Modeling and analysis of reactive multi-component two-phase flows with mass transfer and phase transition the isothermal incompressible case. Discrete and Continuous Dynamical Systems - S, 2017, 10 (4) : 673-696. doi: 10.3934/dcdss.2017034 |
[13] |
Haiyan Yin, Changjiang Zhu. Convergence rate of solutions toward stationary solutions to a viscous liquid-gas two-phase flow model in a half line. Communications on Pure and Applied Analysis, 2015, 14 (5) : 2021-2042. doi: 10.3934/cpaa.2015.14.2021 |
[14] |
Behrouz Kheirfam, Morteza Moslemi. On the extension of an arc-search interior-point algorithm for semidefinite optimization. Numerical Algebra, Control and Optimization, 2018, 8 (2) : 261-275. doi: 10.3934/naco.2018015 |
[15] |
Mohamed A. Tawhid, Ahmed F. Ali. An effective hybrid firefly algorithm with the cuckoo search for engineering optimization problems. Mathematical Foundations of Computing, 2018, 1 (4) : 349-368. doi: 10.3934/mfc.2018017 |
[16] |
M. Soledad Aronna, J. Frédéric Bonnans, Andrei V. Dmitruk, Pablo A. Lotito. Quadratic order conditions for bang-singular extremals. Numerical Algebra, Control and Optimization, 2012, 2 (3) : 511-546. doi: 10.3934/naco.2012.2.511 |
[17] |
Cheng-Ta Yeh, Yi-Kuei Lin. Component allocation cost minimization for a multistate computer network subject to a reliability threshold using tabu search. Journal of Industrial and Management Optimization, 2016, 12 (1) : 141-167. doi: 10.3934/jimo.2016.12.141 |
[18] |
Matthias Gerdts, Stefan Horn, Sven-Joachim Kimmerle. Line search globalization of a semismooth Newton method for operator equations in Hilbert spaces with applications in optimal control. Journal of Industrial and Management Optimization, 2017, 13 (1) : 47-62. doi: 10.3934/jimo.2016003 |
[19] |
Jan Prüss, Jürgen Saal, Gieri Simonett. Singular limits for the two-phase Stefan problem. Discrete and Continuous Dynamical Systems, 2013, 33 (11&12) : 5379-5405. doi: 10.3934/dcds.2013.33.5379 |
[20] |
Theodore Tachim Medjo. A two-phase flow model with delays. Discrete and Continuous Dynamical Systems - B, 2017, 22 (9) : 3273-3294. doi: 10.3934/dcdsb.2017137 |
Impact Factor:
Tools
Metrics
Other articles
by authors
[Back to Top]