American Institute of Mathematical Sciences

Advanced
  • Previous Article
    Distributed convex optimization with coupling constraints over time-varying directed graphs†
  • JIMO Home
  • This Issue
  • Next Article
    Analysis of a batch arrival retrial queue with impatient customers subject to the server disasters
doi: 10.3934/jimo.2020068

Convergence properties of inexact Levenberg-Marquardt method under Hölderian local error bound

Haiyan Wang 1, and  Jinyan Fan 2,,

1. 

School of Mathematical Sciences, Shanghai Jiao Tong University, Shanghai 200240, China
2. 

School of Mathematical Sciences, Shanghai Jiao Tong University, Key Lab of Scientific and Engineering Computing (Ministry of Education), Shanghai 200240, China

* Corresponding author: Jinyan Fan

Received  August 2019 Revised  November 2019 Published  March 2020

Fund Project: The authors are supported by Chinese NSF grants 11971309

In this paper, we study convergence properties of the inexact Levenberg-Marquardt method under the Hölderian local error bound condition and the Hölderian continuity of the Jacobian. The formula of the convergence rates are given, which are functions with respect to the Levenberg-Marquardt parameter, the perturbation vector, as well as the orders of the Hölderian local error bound and Hölderian continuity of the Jacobian.

Keywords: Nonlinear equations, inexact Levenberg-Marquardt method, Hölderian local error bound, Hölderian continuity, convergence rate.
Mathematics Subject Classification: 65K05, 90C30.
Citation: Haiyan Wang, Jinyan Fan. Convergence properties of inexact Levenberg-Marquardt method under Hölderian local error bound. Journal of Industrial & Management Optimization, doi: 10.3934/jimo.2020068
References:
[1]

M. Ahookhosh, F. J. Aragón, R. M. T. Fleming and P. T. Vuong, Local convergence of Levenberg-Marquardt methods under Hölderian metric subregularity, Adv. Comput. Math., 45 (2019), 2771–2806, arXiv: 1703.07461. doi: 10.1007/s10444-019-09708-7.  Google Scholar

[2]

H. Dan, N. Yamashita and M. Fukushima, Convergence properties of the inexact Levenberg-Marquardt method under local error bound, Optimization Methods and Software, 17 (2002), 605-626.  doi: 10.1080/1055678021000049345.  Google Scholar

[3]

F. Facchinei and C. Kanzow, A nonsmooth inexact Newton method for the solution of large-scale nonlinear complementarity problems, Mathematical Programming, 76 (1997), 493-512.  doi: 10.1007/BF02614395.  Google Scholar

[4]

J. Y. Fan, A modified Levenberg-Marquardt algorithm for singular system of nonlinear equations, Journal of Computational Mathematics, 21 (2003), 625-636.   Google Scholar

[5]

J. Y. Fan, The modified Levenberg-Marquardt method for nonlinear equations with cubic convergence, Mathematics of Computation, 81 (2012), 447-466.  doi: 10.1090/S0025-5718-2011-02496-8.  Google Scholar

[6]

J. Y. Fan, J. C. Huang and J. Y. Pan, An adaptive multi-step Levenberg-Marquardt method, Journal of Scientific Computing, 78 (2019), 531-548.  doi: 10.1007/s10915-018-0777-8.  Google Scholar

[7]

J. Y. Fan and J. Y. Pan, Inexact Levenberg-Marquardt method for nonlinear equations, Discrete Continuous Dynamical System-Series B, 4 (2004), 1223-1232.  doi: 10.3934/dcdsb.2004.4.1223.  Google Scholar

[8]

J. Y. Fan and J. Y. Pan, A note on the Levenberg-Marquardt parameter, Applied Mathematics and Computation, 207 (2009), 351-359.  doi: 10.1016/j.amc.2008.10.056.  Google Scholar

[9]

J. Y. Fan and J. Y. Pan, On the convergence rate of the inexact Levenberg-Marquardt method, Industrial and Management Optimization, 7 (2011), 199-210.  doi: 10.3934/jimo.2011.7.199.  Google Scholar

[10]

J. Y. Fan and Y. X. Yuan, On the quadratic convergence of the Levenberg-Marquardt method without nonsingularity assumption, Computing, 74 (2005), 23-39.  doi: 10.1007/s00607-004-0083-1.  Google Scholar

[11]

A. Fischera, P. K. Shuklaa and M. Wang, On the inexactness level of robust Levenberg-Marquardt methods, Optimization, 59 (2010), 273-287.  doi: 10.1080/02331930801951256.  Google Scholar

[12]

C. T. Kelley, Solving Nonlinear Equations with Newton's Method, Fundamentals of Algorithms, SIAM, Philadelphia, 2003. doi: 10.1137/1.9780898718898.  Google Scholar

[13]

K. Levenberg, A method for the solution of certain nonlinear problems in least squares, Quart. Appl. Math., 2 (1944), 164-168.  doi: 10.1090/qam/10666.  Google Scholar

[14]

D. W. Marquardt, An algorithm for least-squares estimation of nonlinear inequalities, SIAM J. Appl. Math., 11 (1963), 431-441.  doi: 10.1137/0111030.  Google Scholar

[15]

J. J. Moré, The Levenberg-Marquardt algorithm: implementation and theory, In: G. A. Watson, ed., Lecture Notes in Mathematics 630: Numerical Analysis, Springer-Verlag, Berlin, 1978, 105–116.  Google Scholar

[16]

M. J. D. Powell, Convergence properties of a class of minimization algorithms, Nonlinear Programming, 2 (1974), 1-27.   Google Scholar

[17]

G. W. Stewart and J.-G. Sun, Matrix Perturbation Theory, (Computer Science and Scientific Computing), Academic Press Boston, 1990.  Google Scholar

[18]

H. Y. Wang and J. Y. Fan, Convergence rate of the Levenberg-Marquardt method under hölderian local error bound, Optimization Methods and Software, 2019. doi: 10.1080/10556788.2019.1694927.  Google Scholar

[19]

N. Yamashita and M. Fukushima, On the rate of convergence of the Levenberg-Marquardt method, Computing, (15) (2001), 239-249.  doi: 10.1007/978-3-7091-6217-0_18.  Google Scholar

[20]

Y. X. Yuan, Recent advances in trust region algorithms, Math. Program., Ser. B, 151 (2015), 249–281. doi: 10.1007/s10107-015-0893-2.  Google Scholar

[21]

X. D. Zhu and G. H. Lin, Improved convergence results for a modified Levenberg-Marquardt method for nonlinear equations and applications in MPCC, Optimization Methods and Software, 31 (2016), 791-804.  doi: 10.1080/10556788.2016.1171863.  Google Scholar

show all references

References:
[1]

M. Ahookhosh, F. J. Aragón, R. M. T. Fleming and P. T. Vuong, Local convergence of Levenberg-Marquardt methods under Hölderian metric subregularity, Adv. Comput. Math., 45 (2019), 2771–2806, arXiv: 1703.07461. doi: 10.1007/s10444-019-09708-7.  Google Scholar

[2]

H. Dan, N. Yamashita and M. Fukushima, Convergence properties of the inexact Levenberg-Marquardt method under local error bound, Optimization Methods and Software, 17 (2002), 605-626.  doi: 10.1080/1055678021000049345.  Google Scholar

[3]

F. Facchinei and C. Kanzow, A nonsmooth inexact Newton method for the solution of large-scale nonlinear complementarity problems, Mathematical Programming, 76 (1997), 493-512.  doi: 10.1007/BF02614395.  Google Scholar

[4]

J. Y. Fan, A modified Levenberg-Marquardt algorithm for singular system of nonlinear equations, Journal of Computational Mathematics, 21 (2003), 625-636.   Google Scholar

[5]

J. Y. Fan, The modified Levenberg-Marquardt method for nonlinear equations with cubic convergence, Mathematics of Computation, 81 (2012), 447-466.  doi: 10.1090/S0025-5718-2011-02496-8.  Google Scholar

[6]

J. Y. Fan, J. C. Huang and J. Y. Pan, An adaptive multi-step Levenberg-Marquardt method, Journal of Scientific Computing, 78 (2019), 531-548.  doi: 10.1007/s10915-018-0777-8.  Google Scholar

[7]

J. Y. Fan and J. Y. Pan, Inexact Levenberg-Marquardt method for nonlinear equations, Discrete Continuous Dynamical System-Series B, 4 (2004), 1223-1232.  doi: 10.3934/dcdsb.2004.4.1223.  Google Scholar

[8]

J. Y. Fan and J. Y. Pan, A note on the Levenberg-Marquardt parameter, Applied Mathematics and Computation, 207 (2009), 351-359.  doi: 10.1016/j.amc.2008.10.056.  Google Scholar

[9]

J. Y. Fan and J. Y. Pan, On the convergence rate of the inexact Levenberg-Marquardt method, Industrial and Management Optimization, 7 (2011), 199-210.  doi: 10.3934/jimo.2011.7.199.  Google Scholar

[10]

J. Y. Fan and Y. X. Yuan, On the quadratic convergence of the Levenberg-Marquardt method without nonsingularity assumption, Computing, 74 (2005), 23-39.  doi: 10.1007/s00607-004-0083-1.  Google Scholar

[11]

A. Fischera, P. K. Shuklaa and M. Wang, On the inexactness level of robust Levenberg-Marquardt methods, Optimization, 59 (2010), 273-287.  doi: 10.1080/02331930801951256.  Google Scholar

[12]

C. T. Kelley, Solving Nonlinear Equations with Newton's Method, Fundamentals of Algorithms, SIAM, Philadelphia, 2003. doi: 10.1137/1.9780898718898.  Google Scholar

[13]

K. Levenberg, A method for the solution of certain nonlinear problems in least squares, Quart. Appl. Math., 2 (1944), 164-168.  doi: 10.1090/qam/10666.  Google Scholar

[14]

D. W. Marquardt, An algorithm for least-squares estimation of nonlinear inequalities, SIAM J. Appl. Math., 11 (1963), 431-441.  doi: 10.1137/0111030.  Google Scholar

[15]

J. J. Moré, The Levenberg-Marquardt algorithm: implementation and theory, In: G. A. Watson, ed., Lecture Notes in Mathematics 630: Numerical Analysis, Springer-Verlag, Berlin, 1978, 105–116.  Google Scholar

[16]

M. J. D. Powell, Convergence properties of a class of minimization algorithms, Nonlinear Programming, 2 (1974), 1-27.   Google Scholar

[17]

G. W. Stewart and J.-G. Sun, Matrix Perturbation Theory, (Computer Science and Scientific Computing), Academic Press Boston, 1990.  Google Scholar

[18]

H. Y. Wang and J. Y. Fan, Convergence rate of the Levenberg-Marquardt method under hölderian local error bound, Optimization Methods and Software, 2019. doi: 10.1080/10556788.2019.1694927.  Google Scholar

[19]

N. Yamashita and M. Fukushima, On the rate of convergence of the Levenberg-Marquardt method, Computing, (15) (2001), 239-249.  doi: 10.1007/978-3-7091-6217-0_18.  Google Scholar

[20]

Y. X. Yuan, Recent advances in trust region algorithms, Math. Program., Ser. B, 151 (2015), 249–281. doi: 10.1007/s10107-015-0893-2.  Google Scholar

[21]

X. D. Zhu and G. H. Lin, Improved convergence results for a modified Levenberg-Marquardt method for nonlinear equations and applications in MPCC, Optimization Methods and Software, 31 (2016), 791-804.  doi: 10.1080/10556788.2016.1171863.  Google Scholar

[1]

Wei Ouyang, Li Li. Hölder strong metric subregularity and its applications to convergence analysis of inexact Newton methods. Journal of Industrial & Management Optimization, 2021, 17 (1) : 169-184. doi: 10.3934/jimo.2019105
[2]

Joel Kübler, Tobias Weth. Spectral asymptotics of radial solutions and nonradial bifurcation for the Hénon equation. Discrete & Continuous Dynamical Systems - A, 2020, 40 (6) : 3629-3656. doi: 10.3934/dcds.2020032
[3]

Hui Gao, Jian Lv, Xiaoliang Wang, Liping Pang. An alternating linearization bundle method for a class of nonconvex optimization problem with inexact information. Journal of Industrial & Management Optimization, 2021, 17 (2) : 805-825. doi: 10.3934/jimo.2019135
[4]

Zuliang Lu, Fei Huang, Xiankui Wu, Lin Li, Shang Liu. Convergence and quasi-optimality of $ L^2- $norms based an adaptive finite element method for nonlinear optimal control problems. Electronic Research Archive, 2020, 28 (4) : 1459-1486. doi: 10.3934/era.2020077
[5]

Patrick Martinez, Judith Vancostenoble. Lipschitz stability for the growth rate coefficients in a nonlinear Fisher-KPP equation. Discrete & Continuous Dynamical Systems - S, 2021, 14 (2) : 695-721. doi: 10.3934/dcdss.2020362
[6]

Hassan Mohammad. A diagonal PRP-type projection method for convex constrained nonlinear monotone equations. Journal of Industrial & Management Optimization, 2021, 17 (1) : 101-116. doi: 10.3934/jimo.2019101
[7]

Alberto Bressan, Wen Shen. A posteriori error estimates for self-similar solutions to the Euler equations. Discrete & Continuous Dynamical Systems - A, 2021, 41 (1) : 113-130. doi: 10.3934/dcds.2020168
[8]

Thierry Cazenave, Ivan Naumkin. Local smooth solutions of the nonlinear Klein-gordon equation. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020448
[9]

Liupeng Wang, Yunqing Huang. Error estimates for second-order SAV finite element method to phase field crystal model. Electronic Research Archive, 2021, 29 (1) : 1735-1752. doi: 10.3934/era.2020089
[10]

Jing Zhou, Cheng Lu, Ye Tian, Xiaoying Tang. A SOCP relaxation based branch-and-bound method for generalized trust-region subproblem. Journal of Industrial & Management Optimization, 2021, 17 (1) : 151-168. doi: 10.3934/jimo.2019104
[11]

Roland Schnaubelt, Martin Spitz. Local wellposedness of quasilinear Maxwell equations with absorbing boundary conditions. Evolution Equations & Control Theory, 2021, 10 (1) : 155-198. doi: 10.3934/eect.2020061
[12]

Gang Luo, Qingzhi Yang. The point-wise convergence of shifted symmetric higher order power method. Journal of Industrial & Management Optimization, 2021, 17 (1) : 357-368. doi: 10.3934/jimo.2019115
[13]

Leilei Wei, Yinnian He. A fully discrete local discontinuous Galerkin method with the generalized numerical flux to solve the tempered fractional reaction-diffusion equation. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020319
[14]

Gunther Uhlmann, Jian Zhai. Inverse problems for nonlinear hyperbolic equations. Discrete & Continuous Dynamical Systems - A, 2021, 41 (1) : 455-469. doi: 10.3934/dcds.2020380
[15]

Xavier Carvajal, Liliana Esquivel, Raphael Santos. On local well-posedness and ill-posedness results for a coupled system of mkdv type equations. Discrete & Continuous Dynamical Systems - A, 2020  doi: 10.3934/dcds.2020382
[16]

Yuxia Guo, Shaolong Peng. A direct method of moving planes for fully nonlinear nonlocal operators and applications. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020462
[17]

Xiu Ye, Shangyou Zhang, Peng Zhu. A weak Galerkin finite element method for nonlinear conservation laws. Electronic Research Archive, 2021, 29 (1) : 1897-1923. doi: 10.3934/era.2020097
[18]

Ke Su, Yumeng Lin, Chun Xu. A new adaptive method to nonlinear semi-infinite programming. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2021012
[19]

Thomas Bartsch, Tian Xu. Strongly localized semiclassical states for nonlinear Dirac equations. Discrete & Continuous Dynamical Systems - A, 2021, 41 (1) : 29-60. doi: 10.3934/dcds.2020297
[20]

Hua Chen, Yawei Wei. Multiple solutions for nonlinear cone degenerate elliptic equations. Communications on Pure & Applied Analysis, , () : -. doi: 10.3934/cpaa.2020272

2019 Impact Factor: 1.366

Tools

Article outline

[Back to Top]

Copyright © 2021 American Institute of Mathematical Sciences