American Institute of Mathematical Sciences

Advanced
October  2013, 9(4): 893-899. doi: 10.3934/jimo.2013.9.893

On the strong convergence of a modified Hestenes-Stiefel method for nonconvex optimization

Weijun Zhou 1, and  Youhua Zhou 1,

1. 

Department of Mathematics, Changsha University of Science and Technology, Changsha 410004, China, China

Received  November 2012 Revised  February 2013 Published  August 2013

In [8], Zhang et al. proposed a modified three-term HS (MTTHS) conjugate gradient method and proved that this method converges globally for nonconvex minimization in the sense that $\liminf_{k\to\infty}\|\nabla f(x_k)\|=0$ when the Armijo or Wolfe line search is used. In this paper, we further study the convergence property of the MTTHS method. We show that the MTTHS method has strongly global convergence property (i.e., $\lim_{k\to\infty}\|\nabla f(x_k)\|=0$) for nonconvex optimization by the use of the backtracking type line search in [7]. Some preliminary numerical results are reported.
Keywords: The MTTHS method, line search, strongly global convergence, $R$-linear convergence..
Mathematics Subject Classification: Primary: 90C30; Secondary: 65K0.
Citation: Weijun Zhou, Youhua Zhou. On the strong convergence of a modified Hestenes-Stiefel method for nonconvex optimization. Journal of Industrial & Management Optimization, 2013, 9 (4) : 893-899. doi: 10.3934/jimo.2013.9.893
References:
[1]

M. R. Hestenes and E. L. Stiefel, Method of conjugate gradient for solving linear systems, J. Res. Nat. Bur. Stand., 49 (1952), 409-432. Google Scholar

[2]

D. Li and M. Fukushima, A modified BFGS method and its global convergence in nonconvex minimization, J. Comput. Appl. Math., 129 (2001), 15-35. doi: 10.1016/S0377-0427(00)00540-9.  Google Scholar

[3]

D. Li and M. Fukushima, On the global convergence of the BFGS method for nonconvex unconstrained optimization problems, SIAM J. Optim., 11 (2001), 1054-1064. doi: 10.1137/S1052623499354242.  Google Scholar

[4]

J. J. Moré, B. S. Garbow and K. H. Hillstrom, Testing unconstrained optimization software, ACM Trans. Math. Softw., 7 (1981), 17-41. doi: 10.1145/355934.355936.  Google Scholar

[5]

E. Polak and G. Ribière, Note sur la convergence de méthodes de directions conjuguées, Rev. Fr. Inform. Rech. Oper., 16 (1969), 35-43.  Google Scholar

[6]

B. T. Polyak, The conjugate gradient method in extreme problems, USSR Comput. Math. Math. Phys., 9 (1969), 94-112. doi: 10.1016/0041-5553(69)90035-4.  Google Scholar

[7]

L. Zhang, W. Zhou and D. Li, A descent modified Polak-Ribière-Polyak conjugate gradient method and its global convergence, IMA J. Numer. Anal., 26 (2006), 629-640. doi: 10.1093/imanum/drl016.  Google Scholar

[8]

L. Zhang, W. Zhou and D. Li, Some descent three-term conjugate gradient methods and their global convergence, Optim. Meth. Softw., 22 (2007), 697-711. doi: 10.1080/10556780701223293.  Google Scholar

[9]

W. Zhou and D. Li, On the convergence properties of the unmodified PRP method with a non-descent line search,, submitted., ().  doi: 10.1080/10556788.2013.811241.  Google Scholar

show all references

References:
[1]

M. R. Hestenes and E. L. Stiefel, Method of conjugate gradient for solving linear systems, J. Res. Nat. Bur. Stand., 49 (1952), 409-432. Google Scholar

[2]

D. Li and M. Fukushima, A modified BFGS method and its global convergence in nonconvex minimization, J. Comput. Appl. Math., 129 (2001), 15-35. doi: 10.1016/S0377-0427(00)00540-9.  Google Scholar

[3]

D. Li and M. Fukushima, On the global convergence of the BFGS method for nonconvex unconstrained optimization problems, SIAM J. Optim., 11 (2001), 1054-1064. doi: 10.1137/S1052623499354242.  Google Scholar

[4]

J. J. Moré, B. S. Garbow and K. H. Hillstrom, Testing unconstrained optimization software, ACM Trans. Math. Softw., 7 (1981), 17-41. doi: 10.1145/355934.355936.  Google Scholar

[5]

E. Polak and G. Ribière, Note sur la convergence de méthodes de directions conjuguées, Rev. Fr. Inform. Rech. Oper., 16 (1969), 35-43.  Google Scholar

[6]

B. T. Polyak, The conjugate gradient method in extreme problems, USSR Comput. Math. Math. Phys., 9 (1969), 94-112. doi: 10.1016/0041-5553(69)90035-4.  Google Scholar

[7]

L. Zhang, W. Zhou and D. Li, A descent modified Polak-Ribière-Polyak conjugate gradient method and its global convergence, IMA J. Numer. Anal., 26 (2006), 629-640. doi: 10.1093/imanum/drl016.  Google Scholar

[8]

L. Zhang, W. Zhou and D. Li, Some descent three-term conjugate gradient methods and their global convergence, Optim. Meth. Softw., 22 (2007), 697-711. doi: 10.1080/10556780701223293.  Google Scholar

[9]

W. Zhou and D. Li, On the convergence properties of the unmodified PRP method with a non-descent line search,, submitted., ().  doi: 10.1080/10556788.2013.811241.  Google Scholar

[1]

Jahnabi Chakravarty, Ashiho Athikho, Manideepa Saha. Convergence of interval AOR method for linear interval equations. Numerical Algebra, Control & Optimization, 2021  doi: 10.3934/naco.2021006
[2]

Yves Bourgault, Damien Broizat, Pierre-Emmanuel Jabin. Convergence rate for the method of moments with linear closure relations. Kinetic & Related Models, 2015, 8 (1) : 1-27. doi: 10.3934/krm.2015.8.1
[3]

Yu-Ning Yang, Su Zhang. On linear convergence of projected gradient method for a class of affine rank minimization problems. Journal of Industrial & Management Optimization, 2016, 12 (4) : 1507-1519. doi: 10.3934/jimo.2016.12.1507
[4]

Leyu Hu, Xingju Cai. Convergence of a randomized Douglas-Rachford method for linear system. Numerical Algebra, Control & Optimization, 2020, 10 (4) : 463-474. doi: 10.3934/naco.2020045
[5]

Wei-Zhe Gu, Li-Yong Lu. The linear convergence of a derivative-free descent method for nonlinear complementarity problems. Journal of Industrial & Management Optimization, 2017, 13 (2) : 531-548. doi: 10.3934/jimo.2016030
[6]

Zhili Ge, Gang Qian, Deren Han. Global convergence of an inexact operator splitting method for monotone variational inequalities. Journal of Industrial & Management Optimization, 2011, 7 (4) : 1013-1026. doi: 10.3934/jimo.2011.7.1013
[7]

Liyan Qi, Xiantao Xiao, Liwei Zhang. On the global convergence of a parameter-adjusting Levenberg-Marquardt method. Numerical Algebra, Control & Optimization, 2015, 5 (1) : 25-36. doi: 10.3934/naco.2015.5.25
[8]

Leong-Kwan Li, Sally Shao. Convergence analysis of the weighted state space search algorithm for recurrent neural networks. Numerical Algebra, Control & Optimization, 2014, 4 (3) : 193-207. doi: 10.3934/naco.2014.4.193
[9]

Zheng-Hai Huang, Nan Lu. Global and global linear convergence of smoothing algorithm for the Cartesian $P_*(\kappa)$-SCLCP. Journal of Industrial & Management Optimization, 2012, 8 (1) : 67-86. doi: 10.3934/jimo.2012.8.67
[10]

Liu Liu. Uniform spectral convergence of the stochastic Galerkin method for the linear semiconductor Boltzmann equation with random inputs and diffusive scaling. Kinetic & Related Models, 2018, 11 (5) : 1139-1156. doi: 10.3934/krm.2018044
[11]

Nora Merabet. Global convergence of a memory gradient method with closed-form step size formula. Conference Publications, 2007, 2007 (Special) : 721-730. doi: 10.3934/proc.2007.2007.721
[12]

Matthias Gerdts, Stefan Horn, Sven-Joachim Kimmerle. Line search globalization of a semismooth Newton method for operator equations in Hilbert spaces with applications in optimal control. Journal of Industrial & Management Optimization, 2017, 13 (1) : 47-62. doi: 10.3934/jimo.2016003
[13]

Wilhelm Schlag. Regularity and convergence rates for the Lyapunov exponents of linear cocycles. Journal of Modern Dynamics, 2013, 7 (4) : 619-637. doi: 10.3934/jmd.2013.7.619
[14]

Feng-Yu Wang. Exponential convergence of non-linear monotone SPDEs. Discrete & Continuous Dynamical Systems, 2015, 35 (11) : 5239-5253. doi: 10.3934/dcds.2015.35.5239
[15]

G. Gentile, V. Mastropietro. Convergence of Lindstedt series for the non linear wave equation. Communications on Pure & Applied Analysis, 2004, 3 (3) : 509-514. doi: 10.3934/cpaa.2004.3.509
[16]

Jinyan Fan, Jianyu Pan. On the convergence rate of the inexact Levenberg-Marquardt method. Journal of Industrial & Management Optimization, 2011, 7 (1) : 199-210. doi: 10.3934/jimo.2011.7.199
[17]

Stefan Kindermann. Convergence of the gradient method for ill-posed problems. Inverse Problems & Imaging, 2017, 11 (4) : 703-720. doi: 10.3934/ipi.2017033
[18]

Karl Kunisch, Markus Müller. Uniform convergence of the POD method and applications to optimal control. Discrete & Continuous Dynamical Systems, 2015, 35 (9) : 4477-4501. doi: 10.3934/dcds.2015.35.4477
[19]

Regina S. Burachik, C. Yalçın Kaya. An update rule and a convergence result for a penalty function method. Journal of Industrial & Management Optimization, 2007, 3 (2) : 381-398. doi: 10.3934/jimo.2007.3.381
[20]

Yong Duan, Jian-Guo Liu. Convergence analysis of the vortex blob method for the $b$-equation. Discrete & Continuous Dynamical Systems, 2014, 34 (5) : 1995-2011. doi: 10.3934/dcds.2014.34.1995

2019 Impact Factor: 1.366

Tools

Metrics

  • PDF downloads (40)
  • HTML views (0)
  • Cited by (2)

Other articles
by authors

[Back to Top]

Copyright © 2021 American Institute of Mathematical Sciences