American Institute of Mathematical Sciences

Advanced
April  2005, 1(2): 181-192. doi: 10.3934/jimo.2005.1.181

Analysis of monotone gradient methods

Yuhong Dai 1, and  Ya-xiang Yuan 2,

1. 

State Key Laboratory of Scientific and Engineering Computing, Institute of Computational Mathematics and Scientific/Engineering Computing, P.O. Box 2719, Beijing 100080, China
2. 

State Key Laboratory of Scientific and Engineering Computing, Institute of Computational Mathematics and Scientific/Engineering computing, Academy of Mathematics and System Sciences, Chinese Academy of Sciences, P. O. Box 2719, Beijing 100080, P. R., China

Received  August 2004 Revised  December 2004 Published  April 2005

The gradient method is one simple method in nonlinear optimization. In this paper, we give a brief review on monotone gradient methods and study their numerical properties by introducing a new technique of long-term observation. We find that, one monotone gradient algorithm which is proposed by Yuan recently shares with the Barzilai-Borwein (BB) method the property that the gradient components with respect to the eigenvectors of the function Hessian are decreasing together. This might partly explain why this algorithm by Yuan is comparable to the BB method in practice. Some examples are also provided showing that the alternate minimization algorithm and the other algorithm by Yuan may fall into cycles. Some more efficient gradient algorithms are provided. Particularly, one of them is monotone and performs better than the BB method in the quadratic case.
Keywords: monotone, cycle., nonmonotone, gradient method, strictly convex quadratics.
Mathematics Subject Classification: 65k05, 90c0.
Citation: Yuhong Dai, Ya-xiang Yuan. Analysis of monotone gradient methods. Journal of Industrial and Management Optimization, 2005, 1 (2) : 181-192. doi: 10.3934/jimo.2005.1.181
[1]

Gaohang Yu, Shanzhou Niu, Jianhua Ma. Multivariate spectral gradient projection method for nonlinear monotone equations with convex constraints. Journal of Industrial and Management Optimization, 2013, 9 (1) : 117-129. doi: 10.3934/jimo.2013.9.117
[2]

Yigui Ou, Yuanwen Liu. A memory gradient method based on the nonmonotone technique. Journal of Industrial and Management Optimization, 2017, 13 (2) : 857-872. doi: 10.3934/jimo.2016050
[3]

Wanbin Tong, Hongjin He, Chen Ling, Liqun Qi. A nonmonotone spectral projected gradient method for tensor eigenvalue complementarity problems. Numerical Algebra, Control and Optimization, 2020, 10 (4) : 425-437. doi: 10.3934/naco.2020042
[4]

Rouhollah Tavakoli, Hongchao Zhang. A nonmonotone spectral projected gradient method for large-scale topology optimization problems. Numerical Algebra, Control and Optimization, 2012, 2 (2) : 395-412. doi: 10.3934/naco.2012.2.395
[5]

Hong Seng Sim, Chuei Yee Chen, Wah June Leong, Jiao Li. Nonmonotone spectral gradient method based on memoryless symmetric rank-one update for large-scale unconstrained optimization. Journal of Industrial and Management Optimization, 2021  doi: 10.3934/jimo.2021143
[6]

Su-Hong Jiang, Min Li. A modified strictly contractive peaceman-rachford splitting method for multi-block separable convex programming. Journal of Industrial and Management Optimization, 2018, 14 (1) : 397-412. doi: 10.3934/jimo.2017052
[7]

Mickaël Crampon. Entropies of strictly convex projective manifolds. Journal of Modern Dynamics, 2009, 3 (4) : 511-547. doi: 10.3934/jmd.2009.3.511
[8]

Jin-Zan Liu, Xin-Wei Liu. A dual Bregman proximal gradient method for relatively-strongly convex optimization. Numerical Algebra, Control and Optimization, 2021  doi: 10.3934/naco.2021028
[9]

Hassan Mohammad. A diagonal PRP-type projection method for convex constrained nonlinear monotone equations. Journal of Industrial and Management Optimization, 2021, 17 (1) : 101-116. doi: 10.3934/jimo.2019101
[10]

Jinkui Liu, Shengjie Li. Multivariate spectral DY-type projection method for convex constrained nonlinear monotone equations. Journal of Industrial and Management Optimization, 2017, 13 (1) : 283-295. doi: 10.3934/jimo.2016017
[11]

Yantao Wang, Linlin Su. Monotone and nonmonotone clines with partial panmixia across a geographical barrier. Discrete and Continuous Dynamical Systems, 2020, 40 (6) : 4019-4037. doi: 10.3934/dcds.2020056
[12]

Lijuan Zhao, Wenyu Sun. Nonmonotone retrospective conic trust region method for unconstrained optimization. Numerical Algebra, Control and Optimization, 2013, 3 (2) : 309-325. doi: 10.3934/naco.2013.3.309
[13]

Joachim Naumann. On the existence of weak solutions of an unsteady p-Laplace thermistor system with strictly monotone electrical conductivities. Discrete and Continuous Dynamical Systems - S, 2017, 10 (4) : 837-852. doi: 10.3934/dcdss.2017042
[14]

Gregorio Díaz, Jesús Ildefonso Díaz. On the free boundary associated with the stationary Monge--Ampère operator on the set of non strictly convex functions. Discrete and Continuous Dynamical Systems, 2015, 35 (4) : 1447-1468. doi: 10.3934/dcds.2015.35.1447
[15]

Martin Heida, Stefan Neukamm, Mario Varga. Stochastic homogenization of $ \Lambda $-convex gradient flows. Discrete and Continuous Dynamical Systems - S, 2021, 14 (1) : 427-453. doi: 10.3934/dcdss.2020328
[16]

Sanming Liu, Zhijie Wang, Chongyang Liu. On convergence analysis of dual proximal-gradient methods with approximate gradient for a class of nonsmooth convex minimization problems. Journal of Industrial and Management Optimization, 2016, 12 (1) : 389-402. doi: 10.3934/jimo.2016.12.389
[17]

Jianjun Zhang, Yunyi Hu, James G. Nagy. A scaled gradient method for digital tomographic image reconstruction. Inverse Problems and Imaging, 2018, 12 (1) : 239-259. doi: 10.3934/ipi.2018010
[18]

José Antonio Carrillo, Yanghong Huang, Francesco Saverio Patacchini, Gershon Wolansky. Numerical study of a particle method for gradient flows. Kinetic and Related Models, 2017, 10 (3) : 613-641. doi: 10.3934/krm.2017025
[19]

Stefan Kindermann. Convergence of the gradient method for ill-posed problems. Inverse Problems and Imaging, 2017, 11 (4) : 703-720. doi: 10.3934/ipi.2017033
[20]

Daniela Saxenhuber, Ronny Ramlau. A gradient-based method for atmospheric tomography. Inverse Problems and Imaging, 2016, 10 (3) : 781-805. doi: 10.3934/ipi.2016021

2020 Impact Factor: 1.801

Tools

Metrics

  • PDF downloads (191)
  • HTML views (0)
  • Cited by (21)

Other articles
by authors

[Back to Top]

Copyright © 2022 American Institute of Mathematical Sciences | Privacy Policy