# American Institute of Mathematical Sciences

March  2021, 11(1): 45-61. doi: 10.3934/naco.2020014

## On the bang-bang control approach via a component-wise line search strategy for unconstrained optimization

 1 School of Mathematical Sciences, Sunway University, Selangor, Malaysia 2 Department of Aerospace and Software Engineering, Gyeongsang National University, Jinju, Republic of Korea 3 School of Electrical Engineering, Computing and Mathematical Sciences, Curtin University, Perth, Australia 4 Department of Electrical and Computer Engineering, Curtin University, Sarawak, Malaysia

* Corresponding author: M. S. LEE (Email: moksianglee@gmail.com)

Received  June 2019 Revised  August 2019 Published  March 2021 Early access  February 2020

A bang-bang iteration method equipped with a component-wise line search strategy is introduced to solve unconstrained optimization problems. The main idea of this method is to formulate an unconstrained optimization problem as an optimal control problem to obtain an optimal trajectory. However, the optimal trajectory can only be generated by impulsive control variables and it is a straight line joining a guessed initial point to a minimum point. Thus, a priori bounds are imposed on the control variables in order to obtain a feasible solution. As a result, the optimal trajectory is made up of bang-bang control sub-arcs, which form an iterative model based on the Lyapunov function's theorem. This is to ensure monotonic decrease of the objective function value and convergence to a desirable minimum point. However, a chattering behavior may occur near the solution. To avoid this behavior, the Newton iterations are then applied to the proposed method via a two-phase approach to achieve fast convergence. Numerical experiments show that this new approach is efficient and cost-effective to solve the unconstrained optimization problems.

Citation: M. S. Lee, H. G. Harno, B. S. Goh, K. H. Lim. On the bang-bang control approach via a component-wise line search strategy for unconstrained optimization. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 45-61. doi: 10.3934/naco.2020014
##### References:

show all references

##### References:
A backtracking procedure to obtain the next iterative point $x(k+1)$ from the current point $x(k)$ on an objective function with two variables
The AGD with rectangular search regions approaching the minimum point $x^{*}$ from the initial point $x(0)$
Performance profiles based on the CPU time
Performance profiles based on the number of iterations
The iterations of the AGDRN method on Booth function (40) in Phase-Ⅰ and Phase-Ⅱ
 Phase-Ⅰ $k$ $f(x)$ $\|g(x)\|$ $\|\Delta x\|$ 0 $3.79\times 10^{3}$ $3.69\times 10^{2}$ $1.80\times 10^{1}$ 1 $3.69\times 10^{2}$ $1.15\times 10^{2}$ $1.41\times 10^{1}$ 2 $3.69\times 10^{2}$ $1.15\times 10^{2}$ $1.27\times 10^{1}$ 3 $2.38\times 10^{1}$ $9.17\times 10^{1}$ $1.15\times 10^{1}$ 4 $4.04\times 10^{1}$ $3.62\times 10^{1}$ $3.09\times 10^{0}$ 5 $1.01\times 10^{1}$ $1.48\times 10^{1}$ $2.78\times 10^{0}$ 6 $4.51\times 10^{0}$ $4.29\times 10^{0}$ $7.52\times 10^{-1}$ 7 $2.10\times 10^{0}$ $2.95\times 10^{0}$ $6.76\times 10^{-1}$ 8 $1.60\times 10^{0}$ $2.60\times 10^{0}$ $1.83\times 10^{-1}$ 9 $1.22\times 10^{0}$ $2.28\times 10^{0}$ $1.64\times 10^{-1}$ 10 $9.13\times 10^{-1}$ $1.99\times 10^{0}$ $1.48\times 10^{-1}$ 11 $6.77\times 10^{-1}$ $1.74\times 10^{0}$ $1.33\times 10^{-1}$ 12 $4.96\times 10^{-1}$ $1.52\times 10^{0}$ $1.20\times 10^{-1}$ 13 $3.57\times 10^{-1}$ $1.33\times 10^{0}$ $1.08\times 10^{-1}$ 14 $2.52\times 10^{-1}$ $1.16\times 10^{0}$ $9.71\times 10^{-2}$ Phase-Ⅱ $j$ $f(x)$ $\|g(x)\|$ {det H} 0 $2.52\times 10^{-1}$ $1.16\times 10^{0}$ $3.60\times 10^{1}$ 1 $4.54\times 10^{-18}$ $4.34\times 10^{-9}$ $3.60\times 10^{1}$
 Phase-Ⅰ $k$ $f(x)$ $\|g(x)\|$ $\|\Delta x\|$ 0 $3.79\times 10^{3}$ $3.69\times 10^{2}$ $1.80\times 10^{1}$ 1 $3.69\times 10^{2}$ $1.15\times 10^{2}$ $1.41\times 10^{1}$ 2 $3.69\times 10^{2}$ $1.15\times 10^{2}$ $1.27\times 10^{1}$ 3 $2.38\times 10^{1}$ $9.17\times 10^{1}$ $1.15\times 10^{1}$ 4 $4.04\times 10^{1}$ $3.62\times 10^{1}$ $3.09\times 10^{0}$ 5 $1.01\times 10^{1}$ $1.48\times 10^{1}$ $2.78\times 10^{0}$ 6 $4.51\times 10^{0}$ $4.29\times 10^{0}$ $7.52\times 10^{-1}$ 7 $2.10\times 10^{0}$ $2.95\times 10^{0}$ $6.76\times 10^{-1}$ 8 $1.60\times 10^{0}$ $2.60\times 10^{0}$ $1.83\times 10^{-1}$ 9 $1.22\times 10^{0}$ $2.28\times 10^{0}$ $1.64\times 10^{-1}$ 10 $9.13\times 10^{-1}$ $1.99\times 10^{0}$ $1.48\times 10^{-1}$ 11 $6.77\times 10^{-1}$ $1.74\times 10^{0}$ $1.33\times 10^{-1}$ 12 $4.96\times 10^{-1}$ $1.52\times 10^{0}$ $1.20\times 10^{-1}$ 13 $3.57\times 10^{-1}$ $1.33\times 10^{0}$ $1.08\times 10^{-1}$ 14 $2.52\times 10^{-1}$ $1.16\times 10^{0}$ $9.71\times 10^{-2}$ Phase-Ⅱ $j$ $f(x)$ $\|g(x)\|$ {det H} 0 $2.52\times 10^{-1}$ $1.16\times 10^{0}$ $3.60\times 10^{1}$ 1 $4.54\times 10^{-18}$ $4.34\times 10^{-9}$ $3.60\times 10^{1}$
The iterations of the AGDRN method on Powell function (41) in Phase-Ⅰ and Phase-Ⅱ
 Phase-Ⅰ $k$ $f(x)$ $\|g(x)\|$ $\|\Delta x\|$ 0 $2.60\times 10^{2}$ $2.53\times 10^{2}$ $5.00\times 10^{0}$ 1 $6.56\times 10^{0}$ $1.31\times 10^{1}$ $3.54\times 10^{0}$ 2 $1.31\times 10^{0}$ $4.51\times 10^{0}$ $2.61\times 10^{0}$ 3 $-2.46\times 10^{-1}$ $9.70\times 10^{-1}$ $1.06\times 10^{0}$ Phase-Ⅱ $j$ $f(x)$ $\|g(x)\|$ {det H} 0 $-2.46\times 10^{-1}$ $9.70\times 10^{-1}$ $5.00\times 10^{-1}$ 1 $-5.49\times 10^{-1}$ $6.63\times 10^{-1}$ $1.36\times 10^{1}$ 2 $-5.82\times 10^{-1}$ $5.20\times 10^{-2}$ $1.09\times 10^{1}$ 3 $-5.82\times 10^{-1}$ $7.61\times 10^{-4}$ $1.06\times 10^{1}$ 4 $-5.82\times 10^{-1}$ $1.71\times 10^{-7}$ $1.06\times 10^{1}$
 Phase-Ⅰ $k$ $f(x)$ $\|g(x)\|$ $\|\Delta x\|$ 0 $2.60\times 10^{2}$ $2.53\times 10^{2}$ $5.00\times 10^{0}$ 1 $6.56\times 10^{0}$ $1.31\times 10^{1}$ $3.54\times 10^{0}$ 2 $1.31\times 10^{0}$ $4.51\times 10^{0}$ $2.61\times 10^{0}$ 3 $-2.46\times 10^{-1}$ $9.70\times 10^{-1}$ $1.06\times 10^{0}$ Phase-Ⅱ $j$ $f(x)$ $\|g(x)\|$ {det H} 0 $-2.46\times 10^{-1}$ $9.70\times 10^{-1}$ $5.00\times 10^{-1}$ 1 $-5.49\times 10^{-1}$ $6.63\times 10^{-1}$ $1.36\times 10^{1}$ 2 $-5.82\times 10^{-1}$ $5.20\times 10^{-2}$ $1.09\times 10^{1}$ 3 $-5.82\times 10^{-1}$ $7.61\times 10^{-4}$ $1.06\times 10^{1}$ 4 $-5.82\times 10^{-1}$ $1.71\times 10^{-7}$ $1.06\times 10^{1}$
Comparison of Phase-Ⅱ replacement with other methods to minimize the Gulf research and development function
 Methods $\boldsymbol{f(x^{*})}$ $\boldsymbol{\|g(x)\|}$ k CPU Time (s) AGDS $5.40\times 10^{-5}$ $6.40\times 10^{-9}$ 37 $9.06\times 10^{-1}$ BTR $5.40\times 10^{-5}$ $5.96\times 10^{-8}$ 81 $1.75\times 10^{0}$ AGDRN $3.11\times 10^{-2}$ $6.61\times 10^{-2}$ 500 $2.15\times 10^{1}$ AGD-RS $5.40\times 10^{-5}$ $1.35\times 10^{-7}$ 83 $1.63\times 10^{0}$
 Methods $\boldsymbol{f(x^{*})}$ $\boldsymbol{\|g(x)\|}$ k CPU Time (s) AGDS $5.40\times 10^{-5}$ $6.40\times 10^{-9}$ 37 $9.06\times 10^{-1}$ BTR $5.40\times 10^{-5}$ $5.96\times 10^{-8}$ 81 $1.75\times 10^{0}$ AGDRN $3.11\times 10^{-2}$ $6.61\times 10^{-2}$ 500 $2.15\times 10^{1}$ AGD-RS $5.40\times 10^{-5}$ $1.35\times 10^{-7}$ 83 $1.63\times 10^{0}$
Comparison of Phase-Ⅱ replacement with other methods to minimize the Kowalik Osborne function
 Methods $\boldsymbol{f(x^{*})}$ $\boldsymbol{\|g(x)\|}$ $\boldsymbol{k}$ CPU Time (s) AGDS $3.08\times 10^{-4}$ $8.60\times 10^{-8}$ 14 $6.09\times 10^{-1}$ BTR $3.08\times 10^{-4}$ $5.76\times 10^{-8}$ 16 $5.94\times 10^{-1}$ AGDRN $5.08\times 10^{-4}$ $3.84\times 10^{-3}$ 499 $2.38\times 10^{1}$ AGD-RS $3.08\times 10^{-4}$ $8.60\times 10^{-8}$ 14 $4.06\times 10^{-1}$
 Methods $\boldsymbol{f(x^{*})}$ $\boldsymbol{\|g(x)\|}$ $\boldsymbol{k}$ CPU Time (s) AGDS $3.08\times 10^{-4}$ $8.60\times 10^{-8}$ 14 $6.09\times 10^{-1}$ BTR $3.08\times 10^{-4}$ $5.76\times 10^{-8}$ 16 $5.94\times 10^{-1}$ AGDRN $5.08\times 10^{-4}$ $3.84\times 10^{-3}$ 499 $2.38\times 10^{1}$ AGD-RS $3.08\times 10^{-4}$ $8.60\times 10^{-8}$ 14 $4.06\times 10^{-1}$
 [1] M. S. Lee, B. S. Goh, H. G. Harno, K. H. Lim. On a two-phase approximate greatest descent method for nonlinear optimization with equality constraints. Numerical Algebra, Control & Optimization, 2018, 8 (3) : 315-326. doi: 10.3934/naco.2018020 [2] Gengsheng Wang, Yubiao Zhang. Decompositions and bang-bang properties. Mathematical Control & Related Fields, 2017, 7 (1) : 73-170. doi: 10.3934/mcrf.2017005 [3] King Hann Lim, Hong Hui Tan, Hendra G. Harno. Approximate greatest descent in neural network optimization. Numerical Algebra, Control & Optimization, 2018, 8 (3) : 327-336. doi: 10.3934/naco.2018021 [4] Helmut Maurer, Tanya Tarnopolskaya, Neale Fulton. Computation of bang-bang and singular controls in collision avoidance. Journal of Industrial & Management Optimization, 2014, 10 (2) : 443-460. doi: 10.3934/jimo.2014.10.443 [5] Karl Kunisch, Lijuan Wang. The bang-bang property of time optimal controls for the Burgers equation. Discrete & Continuous Dynamical Systems, 2014, 34 (9) : 3611-3637. doi: 10.3934/dcds.2014.34.3611 [6] Karl Kunisch, Lijuan Wang. Bang-bang property of time optimal controls of semilinear parabolic equation. Discrete & Continuous Dynamical Systems, 2016, 36 (1) : 279-302. doi: 10.3934/dcds.2016.36.279 [7] Jianjun Liu, Min Zeng, Yifan Ge, Changzhi Wu, Xiangyu Wang. Improved Cuckoo Search algorithm for numerical function optimization. Journal of Industrial & Management Optimization, 2020, 16 (1) : 103-115. doi: 10.3934/jimo.2018142 [8] Walter Alt, Robert Baier, Matthias Gerdts, Frank Lempio. Error bounds for Euler approximation of linear-quadratic control problems with bang-bang solutions. Numerical Algebra, Control & Optimization, 2012, 2 (3) : 547-570. doi: 10.3934/naco.2012.2.547 [9] Zhong Wan, Chaoming Hu, Zhanlu Yang. A spectral PRP conjugate gradient methods for nonconvex optimization problem based on modified line search. Discrete & Continuous Dynamical Systems - B, 2011, 16 (4) : 1157-1169. doi: 10.3934/dcdsb.2011.16.1157 [10] Wataru Nakamura, Yasushi Narushima, Hiroshi Yabe. Nonlinear conjugate gradient methods with sufficient descent properties for unconstrained optimization. Journal of Industrial & Management Optimization, 2013, 9 (3) : 595-619. doi: 10.3934/jimo.2013.9.595 [11] J. Leonel Rocha, Abdel-Kaddous Taha, Danièle Fournier-Prunaret. Explosion birth and extinction: Double big bang bifurcations and Allee effect in Tsoularis-Wallace's growth models. Discrete & Continuous Dynamical Systems - B, 2015, 20 (9) : 3131-3163. doi: 10.3934/dcdsb.2015.20.3131 [12] Dieter Bothe, Jan Prüss. Modeling and analysis of reactive multi-component two-phase flows with mass transfer and phase transition the isothermal incompressible case. Discrete & Continuous Dynamical Systems - S, 2017, 10 (4) : 673-696. doi: 10.3934/dcdss.2017034 [13] Behrouz Kheirfam, Morteza Moslemi. On the extension of an arc-search interior-point algorithm for semidefinite optimization. Numerical Algebra, Control & Optimization, 2018, 8 (2) : 261-275. doi: 10.3934/naco.2018015 [14] Mohamed A. Tawhid, Ahmed F. Ali. An effective hybrid firefly algorithm with the cuckoo search for engineering optimization problems. Mathematical Foundations of Computing, 2018, 1 (4) : 349-368. doi: 10.3934/mfc.2018017 [15] Haiyan Yin, Changjiang Zhu. Convergence rate of solutions toward stationary solutions to a viscous liquid-gas two-phase flow model in a half line. Communications on Pure & Applied Analysis, 2015, 14 (5) : 2021-2042. doi: 10.3934/cpaa.2015.14.2021 [16] M. Soledad Aronna, J. Frédéric Bonnans, Andrei V. Dmitruk, Pablo A. Lotito. Quadratic order conditions for bang-singular extremals. Numerical Algebra, Control & Optimization, 2012, 2 (3) : 511-546. doi: 10.3934/naco.2012.2.511 [17] Cheng-Ta Yeh, Yi-Kuei Lin. Component allocation cost minimization for a multistate computer network subject to a reliability threshold using tabu search. Journal of Industrial & Management Optimization, 2016, 12 (1) : 141-167. doi: 10.3934/jimo.2016.12.141 [18] Matthias Gerdts, Stefan Horn, Sven-Joachim Kimmerle. Line search globalization of a semismooth Newton method for operator equations in Hilbert spaces with applications in optimal control. Journal of Industrial & Management Optimization, 2017, 13 (1) : 47-62. doi: 10.3934/jimo.2016003 [19] Jan Prüss, Jürgen Saal, Gieri Simonett. Singular limits for the two-phase Stefan problem. Discrete & Continuous Dynamical Systems, 2013, 33 (11&12) : 5379-5405. doi: 10.3934/dcds.2013.33.5379 [20] Theodore Tachim Medjo. A two-phase flow model with delays. Discrete & Continuous Dynamical Systems - B, 2017, 22 (9) : 3273-3294. doi: 10.3934/dcdsb.2017137

Impact Factor: