American Institute of Mathematical Sciences

Advanced
October  2013, 9(4): 893-899. doi: 10.3934/jimo.2013.9.893

On the strong convergence of a modified Hestenes-Stiefel method for nonconvex optimization

Weijun Zhou 1, and  Youhua Zhou 1,

1. 

Department of Mathematics, Changsha University of Science and Technology, Changsha 410004, China, China

Received  November 2012 Revised  February 2013 Published  August 2013

In [8], Zhang et al. proposed a modified three-term HS (MTTHS) conjugate gradient method and proved that this method converges globally for nonconvex minimization in the sense that $\liminf_{k\to\infty}\|\nabla f(x_k)\|=0$ when the Armijo or Wolfe line search is used. In this paper, we further study the convergence property of the MTTHS method. We show that the MTTHS method has strongly global convergence property (i.e., $\lim_{k\to\infty}\|\nabla f(x_k)\|=0$) for nonconvex optimization by the use of the backtracking type line search in [7]. Some preliminary numerical results are reported.
Keywords: The MTTHS method, line search, strongly global convergence, $R$-linear convergence..
Mathematics Subject Classification: Primary: 90C30; Secondary: 65K0.
Citation: Weijun Zhou, Youhua Zhou. On the strong convergence of a modified Hestenes-Stiefel method for nonconvex optimization. Journal of Industrial & Management Optimization, 2013, 9 (4) : 893-899. doi: 10.3934/jimo.2013.9.893
References:
[1]

M. R. Hestenes and E. L. Stiefel, Method of conjugate gradient for solving linear systems,, J. Res. Nat. Bur. Stand., 49 (1952), 409.   Google Scholar

[2]

D. Li and M. Fukushima, A modified BFGS method and its global convergence in nonconvex minimization,, J. Comput. Appl. Math., 129 (2001), 15.  doi: 10.1016/S0377-0427(00)00540-9.  Google Scholar

[3]

D. Li and M. Fukushima, On the global convergence of the BFGS method for nonconvex unconstrained optimization problems,, SIAM J. Optim., 11 (2001), 1054.  doi: 10.1137/S1052623499354242.  Google Scholar

[4]

J. J. Moré, B. S. Garbow and K. H. Hillstrom, Testing unconstrained optimization software,, ACM Trans. Math. Softw., 7 (1981), 17.  doi: 10.1145/355934.355936.  Google Scholar

[5]

E. Polak and G. Ribière, Note sur la convergence de méthodes de directions conjuguées,, Rev. Fr. Inform. Rech. Oper., 16 (1969), 35.   Google Scholar

[6]

B. T. Polyak, The conjugate gradient method in extreme problems,, USSR Comput. Math. Math. Phys., 9 (1969), 94.  doi: 10.1016/0041-5553(69)90035-4.  Google Scholar

[7]

L. Zhang, W. Zhou and D. Li, A descent modified Polak-Ribière-Polyak conjugate gradient method and its global convergence,, IMA J. Numer. Anal., 26 (2006), 629.  doi: 10.1093/imanum/drl016.  Google Scholar

[8]

L. Zhang, W. Zhou and D. Li, Some descent three-term conjugate gradient methods and their global convergence,, Optim. Meth. Softw., 22 (2007), 697.  doi: 10.1080/10556780701223293.  Google Scholar

[9]

W. Zhou and D. Li, On the convergence properties of the unmodified PRP method with a non-descent line search,, submitted., ().  doi: 10.1080/10556788.2013.811241.  Google Scholar

show all references

References:
[1]

M. R. Hestenes and E. L. Stiefel, Method of conjugate gradient for solving linear systems,, J. Res. Nat. Bur. Stand., 49 (1952), 409.   Google Scholar

[2]

D. Li and M. Fukushima, A modified BFGS method and its global convergence in nonconvex minimization,, J. Comput. Appl. Math., 129 (2001), 15.  doi: 10.1016/S0377-0427(00)00540-9.  Google Scholar

[3]

D. Li and M. Fukushima, On the global convergence of the BFGS method for nonconvex unconstrained optimization problems,, SIAM J. Optim., 11 (2001), 1054.  doi: 10.1137/S1052623499354242.  Google Scholar

[4]

J. J. Moré, B. S. Garbow and K. H. Hillstrom, Testing unconstrained optimization software,, ACM Trans. Math. Softw., 7 (1981), 17.  doi: 10.1145/355934.355936.  Google Scholar

[5]

E. Polak and G. Ribière, Note sur la convergence de méthodes de directions conjuguées,, Rev. Fr. Inform. Rech. Oper., 16 (1969), 35.   Google Scholar

[6]

B. T. Polyak, The conjugate gradient method in extreme problems,, USSR Comput. Math. Math. Phys., 9 (1969), 94.  doi: 10.1016/0041-5553(69)90035-4.  Google Scholar

[7]

L. Zhang, W. Zhou and D. Li, A descent modified Polak-Ribière-Polyak conjugate gradient method and its global convergence,, IMA J. Numer. Anal., 26 (2006), 629.  doi: 10.1093/imanum/drl016.  Google Scholar

[8]

L. Zhang, W. Zhou and D. Li, Some descent three-term conjugate gradient methods and their global convergence,, Optim. Meth. Softw., 22 (2007), 697.  doi: 10.1080/10556780701223293.  Google Scholar

[9]

W. Zhou and D. Li, On the convergence properties of the unmodified PRP method with a non-descent line search,, submitted., ().  doi: 10.1080/10556788.2013.811241.  Google Scholar

[1]

Haibo Cui, Haiyan Yin. Convergence rate of solutions toward stationary solutions to the isentropic micropolar fluid model in a half line. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020210
[2]

Carlos Gutierrez, Nguyen Van Chau. A remark on an eigenvalue condition for the global injectivity of differentiable maps of $R^2$. Discrete & Continuous Dynamical Systems - A, 2007, 17 (2) : 397-402. doi: 10.3934/dcds.2007.17.397
[3]

J. Frédéric Bonnans, Justina Gianatti, Francisco J. Silva. On the convergence of the Sakawa-Shindo algorithm in stochastic control. Mathematical Control & Related Fields, 2016, 6 (3) : 391-406. doi: 10.3934/mcrf.2016008
[4]

Fernando P. da Costa, João T. Pinto, Rafael Sasportes. On the convergence to critical scaling profiles in submonolayer deposition models. Kinetic & Related Models, 2018, 11 (6) : 1359-1376. doi: 10.3934/krm.2018053
[5]

Alberto Bressan, Carlotta Donadello. On the convergence of viscous approximations after shock interactions. Discrete & Continuous Dynamical Systems - A, 2009, 23 (1&2) : 29-48. doi: 10.3934/dcds.2009.23.29
[6]

Caifang Wang, Tie Zhou. The order of convergence for Landweber Scheme with $\alpha,\beta$-rule. Inverse Problems & Imaging, 2012, 6 (1) : 133-146. doi: 10.3934/ipi.2012.6.133
[7]

Bin Pei, Yong Xu, Yuzhen Bai. Convergence of p-th mean in an averaging principle for stochastic partial differential equations driven by fractional Brownian motion. Discrete & Continuous Dynamical Systems - B, 2020, 25 (3) : 1141-1158. doi: 10.3934/dcdsb.2019213
[8]

Rafael Luís, Sandra Mendonça. A note on global stability in the periodic logistic map. Discrete & Continuous Dynamical Systems - B, 2020, 25 (11) : 4211-4220. doi: 10.3934/dcdsb.2020094
[9]

Lakmi Niwanthi Wadippuli, Ivan Gudoshnikov, Oleg Makarenkov. Global asymptotic stability of nonconvex sweeping processes. Discrete & Continuous Dynamical Systems - B, 2020, 25 (3) : 1129-1139. doi: 10.3934/dcdsb.2019212
[10]

Ademir Fernando Pazoto, Lionel Rosier. Uniform stabilization in weighted Sobolev spaces for the KdV equation posed on the half-line. Discrete & Continuous Dynamical Systems - B, 2010, 14 (4) : 1511-1535. doi: 10.3934/dcdsb.2010.14.1511
[11]

Armin Lechleiter, Tobias Rienmüller. Factorization method for the inverse Stokes problem. Inverse Problems & Imaging, 2013, 7 (4) : 1271-1293. doi: 10.3934/ipi.2013.7.1271
[12]

Bernold Fiedler, Carlos Rocha, Matthias Wolfrum. Sturm global attractors for $S^1$-equivariant parabolic equations. Networks & Heterogeneous Media, 2012, 7 (4) : 617-659. doi: 10.3934/nhm.2012.7.617
[13]

Diana Keller. Optimal control of a linear stochastic Schrödinger equation. Conference Publications, 2013, 2013 (special) : 437-446. doi: 10.3934/proc.2013.2013.437
[14]

Guillaume Bal, Wenjia Jing. Homogenization and corrector theory for linear transport in random media. Discrete & Continuous Dynamical Systems - A, 2010, 28 (4) : 1311-1343. doi: 10.3934/dcds.2010.28.1311
[15]

Qiang Guo, Dong Liang. An adaptive wavelet method and its analysis for parabolic equations. Numerical Algebra, Control & Optimization, 2013, 3 (2) : 327-345. doi: 10.3934/naco.2013.3.327
[16]

Alexander A. Davydov, Massimo Giulietti, Stefano Marcugini, Fernanda Pambianco. Linear nonbinary covering codes and saturating sets in projective spaces. Advances in Mathematics of Communications, 2011, 5 (1) : 119-147. doi: 10.3934/amc.2011.5.119
[17]

W. Cary Huffman. On the theory of $\mathbb{F}_q$-linear $\mathbb{F}_{q^t}$-codes. Advances in Mathematics of Communications, 2013, 7 (3) : 349-378. doi: 10.3934/amc.2013.7.349
[18]

Luke Finlay, Vladimir Gaitsgory, Ivan Lebedev. Linear programming solutions of periodic optimization problems: approximation of the optimal control. Journal of Industrial & Management Optimization, 2007, 3 (2) : 399-413. doi: 10.3934/jimo.2007.3.399
[19]

Irena PawŃow, Wojciech M. Zajączkowski. Global regular solutions to three-dimensional thermo-visco-elasticity with nonlinear temperature-dependent specific heat. Communications on Pure & Applied Analysis, 2017, 16 (4) : 1331-1372. doi: 10.3934/cpaa.2017065
[20]

Lei Liu, Li Wu. Multiplicity of closed characteristics on $ P $-symmetric compact convex hypersurfaces in $ \mathbb{R}^{2n} $. Discrete & Continuous Dynamical Systems - A, 2020  doi: 10.3934/dcds.2020378

2019 Impact Factor: 1.366

Tools

Metrics

  • PDF downloads (36)
  • HTML views (0)
  • Cited by (2)

Other articles
by authors

[Back to Top]

Copyright © 2021 American Institute of Mathematical Sciences