-
Previous Article
Stabilization on input time-varying delay for linear switched systems with truncated predictor control
- NACO Home
- This Issue
-
Next Article
Optimal control of an HIV model with CTL cells and latently infected cells
A new type of quasi-newton updating formulas based on the new quasi-newton equation
Department of Mathematics, College of Computers Sciences and Mathematics, University of Mosul, Iraq |
The quasi-Newton equation is the very foundation of an assortment of the quasi-Newton methods. Therefore, by using the offered alternative equation, we derive the modified BFGS quasi-Newton updating formulas. In this paper, a new y-technique has been introduced to modify the secant equation of the quasi-Newton methods. Prove the global convergence of this algorithm is associated with a line search rule. The numerical results explain that the offered method is effectual for the known test problems.
References:
[1] |
R. Byrd and J. Nocedal,
A tool for the analysis of quasi-Newton methods with application to unconstrained minimization, SIAM J. Numer., 26 (1989), 727-739.
doi: 10.1137/0726042. |
[2] |
X. W. Fang, Q. Ni and M. L. Zeng,
A modified quasi-Newton method for nonlinear equation, Journal of Computational and Applied Mathematics, 328 (2018), 44-58.
doi: 10.1016/j.cam.2017.06.024. |
[3] |
R. Fletcher, Practical Methods of Optimization, John Wiley and Sons, ChiChester, New York, 1987. |
[4] |
A. R. M. Issam,
A new limited memory Quasi-Newton method for unconstrained optimization, J. KSIAM, 7 (2003), 7-14.
|
[5] |
D. Li and M. Fukushima,
A modified BFGS method and its global convergence in nonconvex minimization, J. Comput. Appl. Math., 129 (2001), 15-35.
doi: 10.1016/S0377-0427(00)00540-9. |
[6] |
J. More, B. Garbow and K. Hillstrome,
Testing unconstrained optimization software, ACM Trans. Math. Software, 7 (1981), 17-41.
doi: 10.1145/355934.355936. |
[7] |
J. Nocedal and J. Wright, Numerical Optimization, Springer Series in Operations Research, Springer Verlag, New York, USA.
doi: 10.1007/b98874. |
[8] |
M. J. D. Powell,
Algorithms for nonlinear constraints that use Lagrange functions, Math. Programming, 14 (1978), 224-248.
doi: 10.1007/BF01588967. |
[9] |
Z. Wei, G. Li and L. Qi,
New quasi-Newton methods for unconstrained optimization problems, Math Program. Applied Mathematics and Computation, 175 (2006), 1156-1188.
doi: 10.1016/j.amc.2005.08.027. |
[10] |
Z. Wei, G. Li and L. Qi,
The superlinear convergence of a modified BFGS- type method for unconstrained optimization, Comput. Optim. Appl., 29 (2004), 315-332.
doi: 10.1023/B:COAP.0000044184.25410.39. |
[11] |
P. Wolfe,
Convergence conditions for ascent methods, (Ⅱ): Some corrections, SIAM Review, 13 (1971), 185-188.
doi: 10.1137/1013035. |
[12] |
Y. H. Xiao, Z. X. Wei and L. Zhang,
A modified BFGS method without line searches for nonconvex unconstrained optimization, Advances in Theoretical and Applied Mathematics, 1 (2006), 149-162.
|
[13] |
Y. Yuan and W. Sun, Theory and Methods of Optimization, Science Press of China, 1999. |
[14] |
G. Yuan and Z. Wei,
Convergence analysis of a modified BFGS method on convex minimizations, Comp. Optim. Appl., 47 (2010), 237-255.
doi: 10.1007/s10589-008-9219-0. |
[15] |
G. Yuan, Z. Wei and X. Lu,
Global convergence of BFGS and PRP methods under a modified weak Wolfe-Powell line search, Applied Mathematical Modelling, 47 (2017), 811-825.
doi: 10.1016/j.apm.2017.02.008. |
[16] |
G. Yuan, Z. Sheng, B. Wang, W. Hu and C. Li,
The global convergence of a modified BFGS method for nonconvex functions, Journal of Computational and Applied Mathematics, 327 (2018), 274-294.
doi: 10.1016/j.cam.2017.05.030. |
[17] |
G. Yuan, Z. Wei and Y. Wu,
Modified limited memory BFGS method with nonmonotone line search for unconstrained optimization, J. Korean Math. Soc., 47 (2010), 767-788.
doi: 10.4134/JKMS.2010.47.4.767. |
[18] |
J. Z. Zhang, N. Y. Deng and L. H. Chen,
Quasi-Newton equation and related methods for unconstrained optimization, JOTA, 102 (1999), 147-167.
doi: 10.1023/A:1021898630001. |
show all references
References:
[1] |
R. Byrd and J. Nocedal,
A tool for the analysis of quasi-Newton methods with application to unconstrained minimization, SIAM J. Numer., 26 (1989), 727-739.
doi: 10.1137/0726042. |
[2] |
X. W. Fang, Q. Ni and M. L. Zeng,
A modified quasi-Newton method for nonlinear equation, Journal of Computational and Applied Mathematics, 328 (2018), 44-58.
doi: 10.1016/j.cam.2017.06.024. |
[3] |
R. Fletcher, Practical Methods of Optimization, John Wiley and Sons, ChiChester, New York, 1987. |
[4] |
A. R. M. Issam,
A new limited memory Quasi-Newton method for unconstrained optimization, J. KSIAM, 7 (2003), 7-14.
|
[5] |
D. Li and M. Fukushima,
A modified BFGS method and its global convergence in nonconvex minimization, J. Comput. Appl. Math., 129 (2001), 15-35.
doi: 10.1016/S0377-0427(00)00540-9. |
[6] |
J. More, B. Garbow and K. Hillstrome,
Testing unconstrained optimization software, ACM Trans. Math. Software, 7 (1981), 17-41.
doi: 10.1145/355934.355936. |
[7] |
J. Nocedal and J. Wright, Numerical Optimization, Springer Series in Operations Research, Springer Verlag, New York, USA.
doi: 10.1007/b98874. |
[8] |
M. J. D. Powell,
Algorithms for nonlinear constraints that use Lagrange functions, Math. Programming, 14 (1978), 224-248.
doi: 10.1007/BF01588967. |
[9] |
Z. Wei, G. Li and L. Qi,
New quasi-Newton methods for unconstrained optimization problems, Math Program. Applied Mathematics and Computation, 175 (2006), 1156-1188.
doi: 10.1016/j.amc.2005.08.027. |
[10] |
Z. Wei, G. Li and L. Qi,
The superlinear convergence of a modified BFGS- type method for unconstrained optimization, Comput. Optim. Appl., 29 (2004), 315-332.
doi: 10.1023/B:COAP.0000044184.25410.39. |
[11] |
P. Wolfe,
Convergence conditions for ascent methods, (Ⅱ): Some corrections, SIAM Review, 13 (1971), 185-188.
doi: 10.1137/1013035. |
[12] |
Y. H. Xiao, Z. X. Wei and L. Zhang,
A modified BFGS method without line searches for nonconvex unconstrained optimization, Advances in Theoretical and Applied Mathematics, 1 (2006), 149-162.
|
[13] |
Y. Yuan and W. Sun, Theory and Methods of Optimization, Science Press of China, 1999. |
[14] |
G. Yuan and Z. Wei,
Convergence analysis of a modified BFGS method on convex minimizations, Comp. Optim. Appl., 47 (2010), 237-255.
doi: 10.1007/s10589-008-9219-0. |
[15] |
G. Yuan, Z. Wei and X. Lu,
Global convergence of BFGS and PRP methods under a modified weak Wolfe-Powell line search, Applied Mathematical Modelling, 47 (2017), 811-825.
doi: 10.1016/j.apm.2017.02.008. |
[16] |
G. Yuan, Z. Sheng, B. Wang, W. Hu and C. Li,
The global convergence of a modified BFGS method for nonconvex functions, Journal of Computational and Applied Mathematics, 327 (2018), 274-294.
doi: 10.1016/j.cam.2017.05.030. |
[17] |
G. Yuan, Z. Wei and Y. Wu,
Modified limited memory BFGS method with nonmonotone line search for unconstrained optimization, J. Korean Math. Soc., 47 (2010), 767-788.
doi: 10.4134/JKMS.2010.47.4.767. |
[18] |
J. Z. Zhang, N. Y. Deng and L. H. Chen,
Quasi-Newton equation and related methods for unconstrained optimization, JOTA, 102 (1999), 147-167.
doi: 10.1023/A:1021898630001. |
Author(s) | QN conditions | Ref. |
Powell | [8] | |
Li and Fukushima | [5] | |
Wei, Li, and Qi | [9] | |
Zhang, Deng, and Chen | [18] | |
Yuan and Wei | [14] | |
Yuan, Wei and Wu | [17] |
Author(s) | QN conditions | Ref. |
Powell | [8] | |
Li and Fukushima | [5] | |
Wei, Li, and Qi | [9] | |
Zhang, Deng, and Chen | [18] | |
Yuan and Wei | [14] | |
Yuan, Wei and Wu | [17] |
P.No. | n | BFGS algorithm | BBFGS with |
BBFGS with |
|||
NI | NF | NI | NF | NI | NF | ||
1 | 2 | 35 | 140 | 36 | 124 | 8 | 29 |
2 | 2 | 9 | 26 | 8 | 23 | 5 | 16 |
3 | 2 | 43 | 166 | 34 | 123 | 3 | 12 |
4 | 2 | 3 | 30 | 3 | 30 | 3 | 30 |
5 | 2 | 15 | 50 | 15 | 48 | 5 | 17 |
6 | 2 | 2 | 27 | 2 | 27 | 2 | 27 |
7 | 3 | 34 | 113 | 26 | 86 | 7 | 20 |
8 | 3 | 16 | 54 | 15 | 51 | 6 | 18 |
9 | 3 | 2 | 4 | 2 | 4 | 2 | 4 |
10 | 3 | 2 | 27 | 2 | 27 | 2 | 27 |
11 | 3 | 2 | 27 | 2 | 27 | 2 | 27 |
12 | 4 | 20 | 60 | 20 | 60 | 5 | 17 |
13 | 4 | 19 | 61 | 24 | 73 | 4 | 13 |
14 | 4 | 21 | 65 | 23 | 72 | 4 | 10 |
15 | 4 | 17 | 54 | 16 | 49 | 5 | 17 |
16 | 5 | 2 | 27 | 2 | 27 | 2 | 27 |
17 | 6 | 25 | 72 | 33 | 101 | 4 | 12 |
18 | 11 | 3 | 31 | 3 | 31 | 3 | 31 |
19 | 20 | 31 | 102 | 33 | 103 | 4 | 13 |
20 | 400 | 64 | 209 | 91 | 297 | 5 | 17 |
21 | 400 | 2 | 27 | 2 | 27 | 2 | 27 |
22 | 200 | 2 | 5 | 2 | 5 | 2 | 5 |
23 | 100 | 2 | 27 | 2 | 27 | 2 | 27 |
24 | 500 | 9 | 33 | 8 | 28 | 10 | 31 |
25 | 500 | 2 | 4 | 2 | 4 | 2 | 4 |
26 | 500 | 6 | 16 | 7 | 19 | 5 | 14 |
27 | 500 | 57 | 281 | 16 | 114 | 5 | 17 |
28 | 500 | 2 | 4 | 2 | 4 | 2 | 4 |
29 | 500 | 3 | 7 | 3 | 7 | 3 | 7 |
30 | 500 | 3 | 7 | 3 | 7 | 3 | 7 |
Total | 453 | 1756 | 437 | 1625 | 117 | 527 |
P.No. | n | BFGS algorithm | BBFGS with |
BBFGS with |
|||
NI | NF | NI | NF | NI | NF | ||
1 | 2 | 35 | 140 | 36 | 124 | 8 | 29 |
2 | 2 | 9 | 26 | 8 | 23 | 5 | 16 |
3 | 2 | 43 | 166 | 34 | 123 | 3 | 12 |
4 | 2 | 3 | 30 | 3 | 30 | 3 | 30 |
5 | 2 | 15 | 50 | 15 | 48 | 5 | 17 |
6 | 2 | 2 | 27 | 2 | 27 | 2 | 27 |
7 | 3 | 34 | 113 | 26 | 86 | 7 | 20 |
8 | 3 | 16 | 54 | 15 | 51 | 6 | 18 |
9 | 3 | 2 | 4 | 2 | 4 | 2 | 4 |
10 | 3 | 2 | 27 | 2 | 27 | 2 | 27 |
11 | 3 | 2 | 27 | 2 | 27 | 2 | 27 |
12 | 4 | 20 | 60 | 20 | 60 | 5 | 17 |
13 | 4 | 19 | 61 | 24 | 73 | 4 | 13 |
14 | 4 | 21 | 65 | 23 | 72 | 4 | 10 |
15 | 4 | 17 | 54 | 16 | 49 | 5 | 17 |
16 | 5 | 2 | 27 | 2 | 27 | 2 | 27 |
17 | 6 | 25 | 72 | 33 | 101 | 4 | 12 |
18 | 11 | 3 | 31 | 3 | 31 | 3 | 31 |
19 | 20 | 31 | 102 | 33 | 103 | 4 | 13 |
20 | 400 | 64 | 209 | 91 | 297 | 5 | 17 |
21 | 400 | 2 | 27 | 2 | 27 | 2 | 27 |
22 | 200 | 2 | 5 | 2 | 5 | 2 | 5 |
23 | 100 | 2 | 27 | 2 | 27 | 2 | 27 |
24 | 500 | 9 | 33 | 8 | 28 | 10 | 31 |
25 | 500 | 2 | 4 | 2 | 4 | 2 | 4 |
26 | 500 | 6 | 16 | 7 | 19 | 5 | 14 |
27 | 500 | 57 | 281 | 16 | 114 | 5 | 17 |
28 | 500 | 2 | 4 | 2 | 4 | 2 | 4 |
29 | 500 | 3 | 7 | 3 | 7 | 3 | 7 |
30 | 500 | 3 | 7 | 3 | 7 | 3 | 7 |
Total | 453 | 1756 | 437 | 1625 | 117 | 527 |
BFGS algorithm | BBFGS with |
BBFGS with |
|
NI | 100% | 96.70% | 25.82% |
NF | 100% | 92.53% | 30.01% |
BFGS algorithm | BBFGS with |
BBFGS with |
|
NI | 100% | 96.70% | 25.82% |
NF | 100% | 92.53% | 30.01% |
[1] |
Shummin Nakayama, Yasushi Narushima, Hiroshi Yabe. Memoryless quasi-Newton methods based on spectral-scaling Broyden family for unconstrained optimization. Journal of Industrial and Management Optimization, 2019, 15 (4) : 1773-1793. doi: 10.3934/jimo.2018122 |
[2] |
Yuhong Dai, Nobuo Yamashita. Convergence analysis of sparse quasi-Newton updates with positive definite matrix completion for two-dimensional functions. Numerical Algebra, Control and Optimization, 2011, 1 (1) : 61-69. doi: 10.3934/naco.2011.1.61 |
[3] |
Honglan Zhu, Qin Ni, Meilan Zeng. A quasi-Newton trust region method based on a new fractional model. Numerical Algebra, Control and Optimization, 2015, 5 (3) : 237-249. doi: 10.3934/naco.2015.5.237 |
[4] |
Min Li. A three term Polak-Ribière-Polyak conjugate gradient method close to the memoryless BFGS quasi-Newton method. Journal of Industrial and Management Optimization, 2020, 16 (1) : 245-260. doi: 10.3934/jimo.2018149 |
[5] |
Zohre Aminifard, Saman Babaie-Kafaki. Diagonally scaled memoryless quasi–Newton methods with application to compressed sensing. Journal of Industrial and Management Optimization, 2021 doi: 10.3934/jimo.2021191 |
[6] |
B. S. Goh, W. J. Leong, Z. Siri. Partial Newton methods for a system of equations. Numerical Algebra, Control and Optimization, 2013, 3 (3) : 463-469. doi: 10.3934/naco.2013.3.463 |
[7] |
Cheng-Dar Liou. Note on "Cost analysis of the M/M/R machine repair problem with second optional repair: Newton-Quasi method". Journal of Industrial and Management Optimization, 2012, 8 (3) : 727-732. doi: 10.3934/jimo.2012.8.727 |
[8] |
Kuo-Hsiung Wang, Chuen-Wen Liao, Tseng-Chang Yen. Cost analysis of the M/M/R machine repair problem with second optional repair: Newton-Quasi method. Journal of Industrial and Management Optimization, 2010, 6 (1) : 197-207. doi: 10.3934/jimo.2010.6.197 |
[9] |
Wei Ouyang, Li Li. Hölder strong metric subregularity and its applications to convergence analysis of inexact Newton methods. Journal of Industrial and Management Optimization, 2021, 17 (1) : 169-184. doi: 10.3934/jimo.2019105 |
[10] |
Ai-Li Yang, Yu-Jiang Wu. Newton-MHSS methods for solving systems of nonlinear equations with complex symmetric Jacobian matrices. Numerical Algebra, Control and Optimization, 2012, 2 (4) : 839-853. doi: 10.3934/naco.2012.2.839 |
[11] |
Enrique Fernández-Cara, Arnaud Münch. Numerical null controllability of semi-linear 1-D heat equations: Fixed point, least squares and Newton methods. Mathematical Control and Related Fields, 2012, 2 (3) : 217-246. doi: 10.3934/mcrf.2012.2.217 |
[12] |
Lori Badea. Multigrid methods for some quasi-variational inequalities. Discrete and Continuous Dynamical Systems - S, 2013, 6 (6) : 1457-1471. doi: 10.3934/dcdss.2013.6.1457 |
[13] |
Zhengguang Guo, Sadek Gala. Regularity criterion of the Newton-Boussinesq equations in $R^3$. Communications on Pure and Applied Analysis, 2012, 11 (2) : 443-451. doi: 10.3934/cpaa.2012.11.443 |
[14] |
Xiaojiao Tong, Shuzi Zhou. A smoothing projected Newton-type method for semismooth equations with bound constraints. Journal of Industrial and Management Optimization, 2005, 1 (2) : 235-250. doi: 10.3934/jimo.2005.1.235 |
[15] |
Saeed Ketabchi, Hossein Moosaei, M. Parandegan, Hamidreza Navidi. Computing minimum norm solution of linear systems of equations by the generalized Newton method. Numerical Algebra, Control and Optimization, 2017, 7 (2) : 113-119. doi: 10.3934/naco.2017008 |
[16] |
Qiumei Huang, Xiuxiu Xu, Hermann Brunner. Continuous Galerkin methods on quasi-geometric meshes for delay differential equations of pantograph type. Discrete and Continuous Dynamical Systems, 2016, 36 (10) : 5423-5443. doi: 10.3934/dcds.2016039 |
[17] |
Ugo Locatelli, Letizia Stefanelli. Quasi-periodic motions in a special class of dynamical equations with dissipative effects: A pair of detection methods. Discrete and Continuous Dynamical Systems - B, 2015, 20 (4) : 1155-1187. doi: 10.3934/dcdsb.2015.20.1155 |
[18] |
Li-Xia Liu, Sanyang Liu, Chun-Feng Wang. Smoothing Newton methods for symmetric cone linear complementarity problem with the Cartesian $P$/$P_0$-property. Journal of Industrial and Management Optimization, 2011, 7 (1) : 53-66. doi: 10.3934/jimo.2011.7.53 |
[19] |
Hongxiu Zhong, Guoliang Chen, Xueping Guo. Semi-local convergence of the Newton-HSS method under the center Lipschitz condition. Numerical Algebra, Control and Optimization, 2019, 9 (1) : 85-99. doi: 10.3934/naco.2019007 |
[20] |
Liping Zhang, Soon-Yi Wu, Shu-Cherng Fang. Convergence and error bound of a D-gap function based Newton-type algorithm for equilibrium problems. Journal of Industrial and Management Optimization, 2010, 6 (2) : 333-346. doi: 10.3934/jimo.2010.6.333 |
Impact Factor:
Tools
Metrics
Other articles
by authors
[Back to Top]