Figure 1.
Performance Profiling for the Modified Multiple Spectral Gradient Methods and Standard Multiple Spectral Gradient Method in terms of Number of Iterations.
- Previous Article
- NACO Home
- This Issue
-
Next Article
Differential evolution with improved sub-route reversal repair mechanism for multiobjective urban transit routing problem
September
2018, 8(3): 377-387.
doi: 10.3934/naco.2018024
Multi-step spectral gradient methods with modified weak secant relation for large scale unconstrained optimization
| 1. | Department of Mathematical and Actuarial Sciences, Lee Kong Chian Faculty of Engineering and Science, Universiti Tunku Abdul Rahman, Sungai Long Campus, Jalan Sungai Long 9, Bandar Sungai Long, 43000 Kajang, Selangor, Malaysia |
| 2. | Department of Mathematics, Faculty of Science, Universiti Putra Malaysia, 43400 Serdang, Selangor, Malaysia |
| 3. | Institute for Mathematical Research, Universiti Putra Malaysia, 43400 Serdang, Selangor, Malaysia |
In this paper, we aim to propose some spectral gradient methods via variational technique under log-determinant norm. The spectral parameters satisfy the modified weak secant relations that inspired by the multistep approximation for solving large scale unconstrained optimization. An executable code is developed to test the efficiency of the proposed method with spectral gradient method using standard weak secant relation as constraint. Numerical results are presented which suggest a better performance has been achieved.
Keywords:
Modified weak secant relation,
Spectral gradient methods,
Log-determinant norm,
Large scale,
Unconstrained optimization.
Mathematics Subject Classification:
Primary: 65K10, 93B40; Secondary: 90C06.
Citation:
Hong Seng Sim, Wah June Leong, Chuei Yee Chen, Siti Nur Iqmal Ibrahim. Multi-step spectral gradient methods with modified weak secant relation for large scale unconstrained optimization. Numerical Algebra, Control and Optimization,
2018, 8
(3)
: 377-387.
doi: 10.3934/naco.2018024
![]()
References:
| [1] |
N. Andrei,
An unconstrained optimization test functions collection, Advanced Modeling and Optimization, 10 (2008), 147-161.
|
| [2] |
J. Barzilai and J. Borwein,
Two-point step size gradient methods, IMA Journal of Numerical Analysis, 8 (1988), 141-148.
doi: 10.1093/imanum/8.1.141. |
| [3] |
I. Bongartz, A. R. Conn, N. I. M. Gould and Ph. L. Toint,
CUTE: constrained and unconstrained testing environment, ACM Transactions on Mathematical Software, 21 (1995), 123-160.
doi: 10.1145/200979.201043. |
| [4] |
R. H. Byrd and J. Nocedal,
A tool for the analysis of quasi-Newton methods with application to unconstrained minimization, SIAM J. Numer. Anal., 26 (1989), 727-739.
doi: 10.1137/0726042. |
| [5] |
J. E. Dennis and H. Wolkowicz,
Sizing and least change secant methods, SIAM Journal on Numerical Analysis, 30 (1993), 1291-1313.
doi: 10.1137/0730067. |
| [6] |
J. A. Ford and I. A. Moghrabi,
Multi-step quasi-Newton methods for optimization, Journal of Computational and Applied Mathematics, 50 (1994), 305-323.
doi: 10.1016/0377-0427(94)90309-3. |
| [7] |
W. La Cruz, J. Martinez and M. Raydan,
Spectral residual method without gradient information for solving large-scale nonlinear systems of equations, Math. Comput., 75 (2006), 1449-1466.
doi: 10.1090/S0025-5718-06-01840-0. |
| [8] |
W. La Cruz and M. Raydan,
Nonmonotone spectral methods for large-scale nonlinear systems, Optim. Methods Softw., 18 (2003), 583-599.
doi: 10.1080/10556780310001610493. |
| [9] |
D. C. Liu and J. Nocedal,
On the limited memory BFGS method for large scale optimization, Mathematical Programming, 45 (1989), 503-528.
doi: 10.1007/BF01589116. |
| [10] |
M. Raydan,
The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem, SIAM Journal of Optimization, 7 (1997), 26-33.
doi: 10.1137/S1052623494266365. |
| [11] |
M. Zhu, J. L. Nazareth and H. Wolkowicz,
The quasi-Cauchy relation and diagonal updating, SIAM Journal on Optimization, 4 (1999), 1192-1204.
doi: 10.1137/S1052623498331793. |
show all references
References:
| [1] |
N. Andrei,
An unconstrained optimization test functions collection, Advanced Modeling and Optimization, 10 (2008), 147-161.
|
| [2] |
J. Barzilai and J. Borwein,
Two-point step size gradient methods, IMA Journal of Numerical Analysis, 8 (1988), 141-148.
doi: 10.1093/imanum/8.1.141. |
| [3] |
I. Bongartz, A. R. Conn, N. I. M. Gould and Ph. L. Toint,
CUTE: constrained and unconstrained testing environment, ACM Transactions on Mathematical Software, 21 (1995), 123-160.
doi: 10.1145/200979.201043. |
| [4] |
R. H. Byrd and J. Nocedal,
A tool for the analysis of quasi-Newton methods with application to unconstrained minimization, SIAM J. Numer. Anal., 26 (1989), 727-739.
doi: 10.1137/0726042. |
| [5] |
J. E. Dennis and H. Wolkowicz,
Sizing and least change secant methods, SIAM Journal on Numerical Analysis, 30 (1993), 1291-1313.
doi: 10.1137/0730067. |
| [6] |
J. A. Ford and I. A. Moghrabi,
Multi-step quasi-Newton methods for optimization, Journal of Computational and Applied Mathematics, 50 (1994), 305-323.
doi: 10.1016/0377-0427(94)90309-3. |
| [7] |
W. La Cruz, J. Martinez and M. Raydan,
Spectral residual method without gradient information for solving large-scale nonlinear systems of equations, Math. Comput., 75 (2006), 1449-1466.
doi: 10.1090/S0025-5718-06-01840-0. |
| [8] |
W. La Cruz and M. Raydan,
Nonmonotone spectral methods for large-scale nonlinear systems, Optim. Methods Softw., 18 (2003), 583-599.
doi: 10.1080/10556780310001610493. |
| [9] |
D. C. Liu and J. Nocedal,
On the limited memory BFGS method for large scale optimization, Mathematical Programming, 45 (1989), 503-528.
doi: 10.1007/BF01589116. |
| [10] |
M. Raydan,
The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem, SIAM Journal of Optimization, 7 (1997), 26-33.
doi: 10.1137/S1052623494266365. |
| [11] |
M. Zhu, J. L. Nazareth and H. Wolkowicz,
The quasi-Cauchy relation and diagonal updating, SIAM Journal on Optimization, 4 (1999), 1192-1204.
doi: 10.1137/S1052623498331793. |
Figure 2.
Performance Profiling for the Modified Multiple Spectral Gradient Methods and Standard Multiple Spectral Gradient Method in terms of Number of Function Calls.
Figure 3.
Performance Profiling for the Modified Multiple Spectral Gradient Methods and Standard Multiple Spectral Gradient Method in terms of CPU Time.
| [1] |
Hong Seng Sim, Chuei Yee Chen, Wah June Leong, Jiao Li. Nonmonotone spectral gradient method based on memoryless symmetric rank-one update for large-scale unconstrained optimization. Journal of Industrial and Management Optimization, 2021 doi: 10.3934/jimo.2021143 |
| [2] |
Shishun Li, Zhengda Huang. Guaranteed descent conjugate gradient methods with modified secant condition. Journal of Industrial and Management Optimization, 2008, 4 (4) : 739-755. doi: 10.3934/jimo.2008.4.739 |
| [3] |
Rouhollah Tavakoli, Hongchao Zhang. A nonmonotone spectral projected gradient method for large-scale topology optimization problems. Numerical Algebra, Control and Optimization, 2012, 2 (2) : 395-412. doi: 10.3934/naco.2012.2.395 |
| [4] |
Zhong Wan, Chaoming Hu, Zhanlu Yang. A spectral PRP conjugate gradient methods for nonconvex optimization problem based on modified line search. Discrete and Continuous Dynamical Systems - B, 2011, 16 (4) : 1157-1169. doi: 10.3934/dcdsb.2011.16.1157 |
| [5] |
Wataru Nakamura, Yasushi Narushima, Hiroshi Yabe. Nonlinear conjugate gradient methods with sufficient descent properties for unconstrained optimization. Journal of Industrial and Management Optimization, 2013, 9 (3) : 595-619. doi: 10.3934/jimo.2013.9.595 |
| [6] |
Yigui Ou, Haichan Lin. A class of accelerated conjugate-gradient-like methods based on a modified secant equation. Journal of Industrial and Management Optimization, 2020, 16 (3) : 1503-1518. doi: 10.3934/jimo.2019013 |
| [7] |
Shummin Nakayama, Yasushi Narushima, Hiroshi Yabe. Memoryless quasi-Newton methods based on spectral-scaling Broyden family for unconstrained optimization. Journal of Industrial and Management Optimization, 2019, 15 (4) : 1773-1793. doi: 10.3934/jimo.2018122 |
| [8] |
Sarra Delladji, Mohammed Belloufi, Badreddine Sellami. Behavior of the combination of PRP and HZ methods for unconstrained optimization. Numerical Algebra, Control and Optimization, 2021, 11 (3) : 377-389. doi: 10.3934/naco.2020032 |
| [9] |
Guanghui Zhou, Qin Ni, Meilan Zeng. A scaled conjugate gradient method with moving asymptotes for unconstrained optimization problems. Journal of Industrial and Management Optimization, 2017, 13 (2) : 595-608. doi: 10.3934/jimo.2016034 |
| [10] |
Predrag S. Stanimirović, Branislav Ivanov, Haifeng Ma, Dijana Mosić. A survey of gradient methods for solving nonlinear optimization. Electronic Research Archive, 2020, 28 (4) : 1573-1624. doi: 10.3934/era.2020115 |
| [11] |
Mahmut Çalik, Marcel Oliver. Weak solutions for generalized large-scale semigeostrophic equations. Communications on Pure and Applied Analysis, 2013, 12 (2) : 939-955. doi: 10.3934/cpaa.2013.12.939 |
| [12] |
Mohamed Aly Tawhid. Nonsmooth generalized complementarity as unconstrained optimization. Journal of Industrial and Management Optimization, 2010, 6 (2) : 411-423. doi: 10.3934/jimo.2010.6.411 |
| [13] |
Peter Benner, Ryan Lowe, Matthias Voigt. $\mathcal{L}_{∞}$-norm computation for large-scale descriptor systems using structured iterative eigensolvers. Numerical Algebra, Control and Optimization, 2018, 8 (1) : 119-133. doi: 10.3934/naco.2018007 |
| [14] |
Ya Li, ShouQiang Du, YuanYuan Chen. Modified spectral PRP conjugate gradient method for solving tensor eigenvalue complementarity problems. Journal of Industrial and Management Optimization, 2022, 18 (1) : 157-172. doi: 10.3934/jimo.2020147 |
| [15] |
Liqun Qi, Shenglong Hu, Yanwei Xu. Spectral norm and nuclear norm of a third order tensor. Journal of Industrial and Management Optimization, 2022, 18 (2) : 1101-1113. doi: 10.3934/jimo.2021010 |
| [16] |
Gaohang Yu, Lutai Guan, Guoyin Li. Global convergence of modified Polak-Ribière-Polyak conjugate gradient methods with sufficient descent property. Journal of Industrial and Management Optimization, 2008, 4 (3) : 565-579. doi: 10.3934/jimo.2008.4.565 |
| [17] |
Yigui Ou, Xin Zhou. A modified scaled memoryless BFGS preconditioned conjugate gradient algorithm for nonsmooth convex optimization. Journal of Industrial and Management Optimization, 2018, 14 (2) : 785-801. doi: 10.3934/jimo.2017075 |
| [18] |
Junxiang Li, Yan Gao, Tao Dai, Chunming Ye, Qiang Su, Jiazhen Huo. Substitution secant/finite difference method to large sparse minimax problems. Journal of Industrial and Management Optimization, 2014, 10 (2) : 637-663. doi: 10.3934/jimo.2014.10.637 |
| [19] |
Tengteng Yu, Xin-Wei Liu, Yu-Hong Dai, Jie Sun. Variable metric proximal stochastic variance reduced gradient methods for nonconvex nonsmooth optimization. Journal of Industrial and Management Optimization, 2021 doi: 10.3934/jimo.2021084 |
| [20] |
Yuhong Dai, Ya-xiang Yuan. Analysis of monotone gradient methods. Journal of Industrial and Management Optimization, 2005, 1 (2) : 181-192. doi: 10.3934/jimo.2005.1.181 |
Impact Factor:
Tools
Metrics
Other articles
by authors
[Back to Top]