On the strong convergence of a modified Hestenes-Stiefel method for nonconvex optimization

Previous Article
Equilibrium joining probabilities in observable queues with general service and setup times
JIMO Home
This Issue
Next Article
Reduction and dynamic approach for the multi-choice Shapley value

October 2013, 9(4): 893-899. doi: 10.3934/jimo.2013.9.893

On the strong convergence of a modified Hestenes-Stiefel method for nonconvex optimization

Weijun Zhou ^1, and Youhua Zhou ^1,

1.	Department of Mathematics, Changsha University of Science and Technology, Changsha 410004, China, China

Received November 2012 Revised February 2013 Published August 2013

Abstract
Related Papers

In [8], Zhang et al. proposed a modified three-term HS (MTTHS) conjugate gradient method and proved that this method converges globally for nonconvex minimization in the sense that $\liminf_{k\to\infty}\|\nabla f(x_k)\|=0$ when the Armijo or Wolfe line search is used. In this paper, we further study the convergence property of the MTTHS method. We show that the MTTHS method has strongly global convergence property (i.e., $\lim_{k\to\infty}\|\nabla f(x_k)\|=0$) for nonconvex optimization by the use of the backtracking type line search in [7]. Some preliminary numerical results are reported.

Keywords: The MTTHS method, line search, strongly global convergence, $R$-linear convergence..

Mathematics Subject Classification: Primary: 90C30; Secondary: 65K0.

Citation: Weijun Zhou, Youhua Zhou. On the strong convergence of a modified Hestenes-Stiefel method for nonconvex optimization. Journal of Industrial and Management Optimization, 2013, 9 (4) : 893-899. doi: 10.3934/jimo.2013.9.893

References:

[1]	M. R. Hestenes and E. L. Stiefel, Method of conjugate gradient for solving linear systems, J. Res. Nat. Bur. Stand., 49 (1952), 409-432.
[2]	D. Li and M. Fukushima, A modified BFGS method and its global convergence in nonconvex minimization, J. Comput. Appl. Math., 129 (2001), 15-35. doi: 10.1016/S0377-0427(00)00540-9.
[3]	D. Li and M. Fukushima, On the global convergence of the BFGS method for nonconvex unconstrained optimization problems, SIAM J. Optim., 11 (2001), 1054-1064. doi: 10.1137/S1052623499354242.
[4]	J. J. Moré, B. S. Garbow and K. H. Hillstrom, Testing unconstrained optimization software, ACM Trans. Math. Softw., 7 (1981), 17-41. doi: 10.1145/355934.355936.
[5]	E. Polak and G. Ribière, Note sur la convergence de méthodes de directions conjuguées, Rev. Fr. Inform. Rech. Oper., 16 (1969), 35-43.
[6]	B. T. Polyak, The conjugate gradient method in extreme problems, USSR Comput. Math. Math. Phys., 9 (1969), 94-112. doi: 10.1016/0041-5553(69)90035-4.
[7]	L. Zhang, W. Zhou and D. Li, A descent modified Polak-Ribière-Polyak conjugate gradient method and its global convergence, IMA J. Numer. Anal., 26 (2006), 629-640. doi: 10.1093/imanum/drl016.
[8]	L. Zhang, W. Zhou and D. Li, Some descent three-term conjugate gradient methods and their global convergence, Optim. Meth. Softw., 22 (2007), 697-711. doi: 10.1080/10556780701223293.
[9]	W. Zhou and D. Li, On the convergence properties of the unmodified PRP method with a non-descent line search,, submitted., (). doi: 10.1080/10556788.2013.811241.

show all references

References:

[1]	M. R. Hestenes and E. L. Stiefel, Method of conjugate gradient for solving linear systems, J. Res. Nat. Bur. Stand., 49 (1952), 409-432.
[2]	D. Li and M. Fukushima, A modified BFGS method and its global convergence in nonconvex minimization, J. Comput. Appl. Math., 129 (2001), 15-35. doi: 10.1016/S0377-0427(00)00540-9.
[3]	D. Li and M. Fukushima, On the global convergence of the BFGS method for nonconvex unconstrained optimization problems, SIAM J. Optim., 11 (2001), 1054-1064. doi: 10.1137/S1052623499354242.
[4]	J. J. Moré, B. S. Garbow and K. H. Hillstrom, Testing unconstrained optimization software, ACM Trans. Math. Softw., 7 (1981), 17-41. doi: 10.1145/355934.355936.
[5]	E. Polak and G. Ribière, Note sur la convergence de méthodes de directions conjuguées, Rev. Fr. Inform. Rech. Oper., 16 (1969), 35-43.
[6]	B. T. Polyak, The conjugate gradient method in extreme problems, USSR Comput. Math. Math. Phys., 9 (1969), 94-112. doi: 10.1016/0041-5553(69)90035-4.
[7]	L. Zhang, W. Zhou and D. Li, A descent modified Polak-Ribière-Polyak conjugate gradient method and its global convergence, IMA J. Numer. Anal., 26 (2006), 629-640. doi: 10.1093/imanum/drl016.
[8]	L. Zhang, W. Zhou and D. Li, Some descent three-term conjugate gradient methods and their global convergence, Optim. Meth. Softw., 22 (2007), 697-711. doi: 10.1080/10556780701223293.
[9]	W. Zhou and D. Li, On the convergence properties of the unmodified PRP method with a non-descent line search,, submitted., (). doi: 10.1080/10556788.2013.811241.

[1]	Yves Bourgault, Damien Broizat, Pierre-Emmanuel Jabin. Convergence rate for the method of moments with linear closure relations. Kinetic and Related Models, 2015, 8 (1) : 1-27. doi: 10.3934/krm.2015.8.1
[2]	Jahnabi Chakravarty, Ashiho Athikho, Manideepa Saha. Convergence of interval AOR method for linear interval equations. Numerical Algebra, Control and Optimization, 2022, 12 (2) : 293-308. doi: 10.3934/naco.2021006
[3]	Yu-Ning Yang, Su Zhang. On linear convergence of projected gradient method for a class of affine rank minimization problems. Journal of Industrial and Management Optimization, 2016, 12 (4) : 1507-1519. doi: 10.3934/jimo.2016.12.1507
[4]	Leyu Hu, Xingju Cai. Convergence of a randomized Douglas-Rachford method for linear system. Numerical Algebra, Control and Optimization, 2020, 10 (4) : 463-474. doi: 10.3934/naco.2020045
[5]	Wei-Zhe Gu, Li-Yong Lu. The linear convergence of a derivative-free descent method for nonlinear complementarity problems. Journal of Industrial and Management Optimization, 2017, 13 (2) : 531-548. doi: 10.3934/jimo.2016030
[6]	Zhili Ge, Gang Qian, Deren Han. Global convergence of an inexact operator splitting method for monotone variational inequalities. Journal of Industrial and Management Optimization, 2011, 7 (4) : 1013-1026. doi: 10.3934/jimo.2011.7.1013
[7]	Liyan Qi, Xiantao Xiao, Liwei Zhang. On the global convergence of a parameter-adjusting Levenberg-Marquardt method. Numerical Algebra, Control and Optimization, 2015, 5 (1) : 25-36. doi: 10.3934/naco.2015.5.25
[8]	Gonglin Yuan, Zhan Wang, Pengyuan Li. Global convergence of a modified Broyden family method for nonconvex functions. Journal of Industrial and Management Optimization, 2021 doi: 10.3934/jimo.2021164
[9]	Leong-Kwan Li, Sally Shao. Convergence analysis of the weighted state space search algorithm for recurrent neural networks. Numerical Algebra, Control and Optimization, 2014, 4 (3) : 193-207. doi: 10.3934/naco.2014.4.193
[10]	Zheng-Hai Huang, Nan Lu. Global and global linear convergence of smoothing algorithm for the Cartesian $P_(\kappa)$-SCLCP. Journal of Industrial and Management Optimization*, 2012, 8 (1) : 67-86. doi: 10.3934/jimo.2012.8.67
[11]	Liu Liu. Uniform spectral convergence of the stochastic Galerkin method for the linear semiconductor Boltzmann equation with random inputs and diffusive scaling. Kinetic and Related Models, 2018, 11 (5) : 1139-1156. doi: 10.3934/krm.2018044
[12]	Nora Merabet. Global convergence of a memory gradient method with closed-form step size formula. Conference Publications, 2007, 2007 (Special) : 721-730. doi: 10.3934/proc.2007.2007.721
[13]	Matthias Gerdts, Stefan Horn, Sven-Joachim Kimmerle. Line search globalization of a semismooth Newton method for operator equations in Hilbert spaces with applications in optimal control. Journal of Industrial and Management Optimization, 2017, 13 (1) : 47-62. doi: 10.3934/jimo.2016003
[14]	Wilhelm Schlag. Regularity and convergence rates for the Lyapunov exponents of linear cocycles. Journal of Modern Dynamics, 2013, 7 (4) : 619-637. doi: 10.3934/jmd.2013.7.619
[15]	Feng-Yu Wang. Exponential convergence of non-linear monotone SPDEs. Discrete and Continuous Dynamical Systems, 2015, 35 (11) : 5239-5253. doi: 10.3934/dcds.2015.35.5239
[16]	G. Gentile, V. Mastropietro. Convergence of Lindstedt series for the non linear wave equation. Communications on Pure and Applied Analysis, 2004, 3 (3) : 509-514. doi: 10.3934/cpaa.2004.3.509
[17]	Jinyan Fan, Jianyu Pan. On the convergence rate of the inexact Levenberg-Marquardt method. Journal of Industrial and Management Optimization, 2011, 7 (1) : 199-210. doi: 10.3934/jimo.2011.7.199
[18]	Stefan Kindermann. Convergence of the gradient method for ill-posed problems. Inverse Problems and Imaging, 2017, 11 (4) : 703-720. doi: 10.3934/ipi.2017033
[19]	Karl Kunisch, Markus Müller. Uniform convergence of the POD method and applications to optimal control. Discrete and Continuous Dynamical Systems, 2015, 35 (9) : 4477-4501. doi: 10.3934/dcds.2015.35.4477
[20]	Regina S. Burachik, C. Yalçın Kaya. An update rule and a convergence result for a penalty function method. Journal of Industrial and Management Optimization, 2007, 3 (2) : 381-398. doi: 10.3934/jimo.2007.3.381

2020 Impact Factor: 1.801

Tools

Metrics

Other articles
by authors

on AIMS
Weijun Zhou Youhua Zhou
on Google Scholar
Weijun Zhou Youhua Zhou

[Back to Top]