American Institute of Mathematical Sciences

Advanced
  • Previous Article
    An outcome space algorithm for minimizing the product of two convex functions over a convex set
  • JIMO Home
  • This Issue
  • Next Article
    Applications of a nonlinear optimization solver and two-stage comprehensive Denoising techniques for optimum underwater wideband sonar echolocation system
January  2013, 9(1): 227-241. doi: 10.3934/jimo.2013.9.227

On the Levenberg-Marquardt methods for convex constrained nonlinear equations

Jinyan Fan 1,

1. 

Department of Mathematics, and MOE-LSC, Shanghai Jiao Tong University, Shanghai 200240, China

Received  January 2012 Revised  July 2012 Published  December 2012

In this paper, both the constrained Levenberg-Marquardt method and the projected Levenberg-Marquardt method are presented for nonlinear equations $F(x)=0$ subject to $x\in X$, where $X$ is a nonempty closed convex set. The Levenberg-Marquardt parameter is taken as $\| F(x_k) \|_2^\delta$ with $\delta\in (0, 2]$. Under the local error bound condition which is weaker than nonsingularity, the methods are shown to have the same convergence rate, which includes not only the convergence results obtained in [12] for $\delta=2$ but also the results given in [7] for unconstrained nonlinear equations.
Keywords: Convex constrained nonlinear equations, quadratic convergence., local error bound, constrained Levenberg-Marquardt method, projected Levenberg-Marquardt method.
Mathematics Subject Classification: 90C30, 65K0.
Citation: Jinyan Fan. On the Levenberg-Marquardt methods for convex constrained nonlinear equations. Journal of Industrial & Management Optimization, 2013, 9 (1) : 227-241. doi: 10.3934/jimo.2013.9.227
References:
[1]

S. Bellavia, M. Macconi and B. Morini, An affine scaling trust-region approach to bound-constrained nonlinear systems,, Appl. Numer. Math., 44 (2003), 257.  doi: 10.1016/S0168-9274(02)00170-8.  Google Scholar

[2]

S. Bellavia and B. Morini, An interior global method for nonlinear systems with simple bounds,, Optim. Methods Software, 20 (2005), 1.   Google Scholar

[3]

H. Dan, N. Yamashita and M. Fukushima, Convergence properties of the inexact Levenberg-Marquardt method under local error bound conditions,, Optim. Meth. Software, 17 (2002), 605.   Google Scholar

[4]

J. E. Dennis and R. B. Schnabel, "Numerical Methods for Unconstrained Optimization and Nonlinear Equations,", Prentice-Hall, (1983).   Google Scholar

[5]

S. P. Dirkse, M. C. Ferris, MCPLIB: a collection of nonlinear mixed complementarity problems,, Optim. Meth. Software, 5 (1995), 319.  doi: 10.1080/10556789508805619.  Google Scholar

[6]

M. E. El-Hawary, "Optimal Power Flow: Solutions Techniques, Requirements, and Challenges,", IEEE Service Center, (1996).   Google Scholar

[7]

J. Y. Fan and J. Y. Pan, Convergence properties of a self-adaptive Levenberg-Marquardt algorithm under local error bound condition,, Computational Optimization and Applications, 34 (2006), 47.   Google Scholar

[8]

J. Y. Fan and Y. X. Yuan, On the quadratic convergence of the Levenberg-Marquardt method without nonsingularity assumption,, Computing, 74 (2005), 23.  doi: 10.1007/s00607-004-0083-1.  Google Scholar

[9]

C. A. Floudas, P. M. Pardalos, C. S. Adjiman, W. R. Esposito, Z. H. Gumus, S. T. Harding, J. L. Klepeis, C. A. Meyer and C. A. Schweiger, "Handbook of Test Problems in Local and Global Optimization, Nonconvex Optimization and its Applications,", Vol. 33, (1999).   Google Scholar

[10]

W. Hock and K. Schittkowski, "Test Examples for Nonlinear Programming Codes,", Lecture Notes in Economics and Mathematical Systems, (1981).   Google Scholar

[11]

C. Kanzow, An active set-type Newton method for constrained nonlinear systems,, in, (2001), 179.   Google Scholar

[12]

C. Kanzow, N. Yamashita and M. Fukushima, Levenberg-Marquardt methods with strong local convergence properties for solving nonlinear equations with convex constraints,, Journal of Computational and Applied Mathematics, 173 (2005), 321.   Google Scholar

[13]

C. T. Kelley, "Iterative Methods for Linear and Nonlinear Equations,", SIAM, (1995).   Google Scholar

[14]

D. N. Kozakevich, J. M. Martinez and S. A. Santos, Solving nonlinear systems of equations with simple bounds,, Comput. Appl. Math., 16 (1997), 215.   Google Scholar

[15]

K. Levenberg, A method for the solution of certain nonlinear problems in least squares,, Quart. Appl. Math., 2 (1944), 164.   Google Scholar

[16]

D. W. Marquardt, An algorithm for least-squares estimation of nonlinear inequalities,, SIAM J. Appl. Math., 11 (1963), 431.   Google Scholar

[17]

K. Meintjes and A. P. Morgan, A methodology for solving chemical equilibrium systems,, Appl. Math. Comput., 22 (1987), 333.   Google Scholar

[18]

K. Meintjes and A. P. Morgan, Chemical equilibrium systems as numerical test problems,, ACM Trans. Math. Software, 16 (1990), 143.   Google Scholar

[19]

R. D. C. Monteiro and J. S. Pang, A potential reduction Newton method for constrained equations,, SIAM J. Optim., 9 (1999), 729.  doi: 10.1137/S1052623497318980.  Google Scholar

[20]

J. J. Moré, The LM algorithm: implementation and theory,, in, (1978), 105.   Google Scholar

[21]

J. M. Ortega and W. C. Rheinboldt, "Iterative solution of Nonlinear Equations in Several Variables,", Academic Press, (1970).   Google Scholar

[22]

L. Qi, X. J. Tong and D. H. Li, An active-set projected trust region algorithm for box constrained nonsmooth equations,, Journal of Optimization Theories and Applications, 120 (2004), 601.   Google Scholar

[23]

M. Ulbrich, Nonmonotone trust-region methods for bound-constrained semismooth equations with applications to nonlinear mixed complementarity problems,, SIAM J. Optim., 11 (2001), 889.   Google Scholar

[24]

T. Wang, R. D. C. Monteiro and J. S. Pang, An interior point potential reduction method for constrained equations,, Math. Programming, 74 (1996), 159.   Google Scholar

[25]

A. J. Wood and B. F. Wollenberg, "Power Generation, Operation, and Control,", John Wiley and Sons, (1996).   Google Scholar

[26]

N. Yamashita and M. Fukushima, On the rate of convergence of the LM method,, Computing (Supplement 15), (2001), 237.   Google Scholar

[27]

Y. X. Yuan, Recent advances in numerical methods for nonlinear equations and nonlinear least sqaures,, Numerical Algebra, 1 (2011), 15.   Google Scholar

show all references

References:
[1]

S. Bellavia, M. Macconi and B. Morini, An affine scaling trust-region approach to bound-constrained nonlinear systems,, Appl. Numer. Math., 44 (2003), 257.  doi: 10.1016/S0168-9274(02)00170-8.  Google Scholar

[2]

S. Bellavia and B. Morini, An interior global method for nonlinear systems with simple bounds,, Optim. Methods Software, 20 (2005), 1.   Google Scholar

[3]

H. Dan, N. Yamashita and M. Fukushima, Convergence properties of the inexact Levenberg-Marquardt method under local error bound conditions,, Optim. Meth. Software, 17 (2002), 605.   Google Scholar

[4]

J. E. Dennis and R. B. Schnabel, "Numerical Methods for Unconstrained Optimization and Nonlinear Equations,", Prentice-Hall, (1983).   Google Scholar

[5]

S. P. Dirkse, M. C. Ferris, MCPLIB: a collection of nonlinear mixed complementarity problems,, Optim. Meth. Software, 5 (1995), 319.  doi: 10.1080/10556789508805619.  Google Scholar

[6]

M. E. El-Hawary, "Optimal Power Flow: Solutions Techniques, Requirements, and Challenges,", IEEE Service Center, (1996).   Google Scholar

[7]

J. Y. Fan and J. Y. Pan, Convergence properties of a self-adaptive Levenberg-Marquardt algorithm under local error bound condition,, Computational Optimization and Applications, 34 (2006), 47.   Google Scholar

[8]

J. Y. Fan and Y. X. Yuan, On the quadratic convergence of the Levenberg-Marquardt method without nonsingularity assumption,, Computing, 74 (2005), 23.  doi: 10.1007/s00607-004-0083-1.  Google Scholar

[9]

C. A. Floudas, P. M. Pardalos, C. S. Adjiman, W. R. Esposito, Z. H. Gumus, S. T. Harding, J. L. Klepeis, C. A. Meyer and C. A. Schweiger, "Handbook of Test Problems in Local and Global Optimization, Nonconvex Optimization and its Applications,", Vol. 33, (1999).   Google Scholar

[10]

W. Hock and K. Schittkowski, "Test Examples for Nonlinear Programming Codes,", Lecture Notes in Economics and Mathematical Systems, (1981).   Google Scholar

[11]

C. Kanzow, An active set-type Newton method for constrained nonlinear systems,, in, (2001), 179.   Google Scholar

[12]

C. Kanzow, N. Yamashita and M. Fukushima, Levenberg-Marquardt methods with strong local convergence properties for solving nonlinear equations with convex constraints,, Journal of Computational and Applied Mathematics, 173 (2005), 321.   Google Scholar

[13]

C. T. Kelley, "Iterative Methods for Linear and Nonlinear Equations,", SIAM, (1995).   Google Scholar

[14]

D. N. Kozakevich, J. M. Martinez and S. A. Santos, Solving nonlinear systems of equations with simple bounds,, Comput. Appl. Math., 16 (1997), 215.   Google Scholar

[15]

K. Levenberg, A method for the solution of certain nonlinear problems in least squares,, Quart. Appl. Math., 2 (1944), 164.   Google Scholar

[16]

D. W. Marquardt, An algorithm for least-squares estimation of nonlinear inequalities,, SIAM J. Appl. Math., 11 (1963), 431.   Google Scholar

[17]

K. Meintjes and A. P. Morgan, A methodology for solving chemical equilibrium systems,, Appl. Math. Comput., 22 (1987), 333.   Google Scholar

[18]

K. Meintjes and A. P. Morgan, Chemical equilibrium systems as numerical test problems,, ACM Trans. Math. Software, 16 (1990), 143.   Google Scholar

[19]

R. D. C. Monteiro and J. S. Pang, A potential reduction Newton method for constrained equations,, SIAM J. Optim., 9 (1999), 729.  doi: 10.1137/S1052623497318980.  Google Scholar

[20]

J. J. Moré, The LM algorithm: implementation and theory,, in, (1978), 105.   Google Scholar

[21]

J. M. Ortega and W. C. Rheinboldt, "Iterative solution of Nonlinear Equations in Several Variables,", Academic Press, (1970).   Google Scholar

[22]

L. Qi, X. J. Tong and D. H. Li, An active-set projected trust region algorithm for box constrained nonsmooth equations,, Journal of Optimization Theories and Applications, 120 (2004), 601.   Google Scholar

[23]

M. Ulbrich, Nonmonotone trust-region methods for bound-constrained semismooth equations with applications to nonlinear mixed complementarity problems,, SIAM J. Optim., 11 (2001), 889.   Google Scholar

[24]

T. Wang, R. D. C. Monteiro and J. S. Pang, An interior point potential reduction method for constrained equations,, Math. Programming, 74 (1996), 159.   Google Scholar

[25]

A. J. Wood and B. F. Wollenberg, "Power Generation, Operation, and Control,", John Wiley and Sons, (1996).   Google Scholar

[26]

N. Yamashita and M. Fukushima, On the rate of convergence of the LM method,, Computing (Supplement 15), (2001), 237.   Google Scholar

[27]

Y. X. Yuan, Recent advances in numerical methods for nonlinear equations and nonlinear least sqaures,, Numerical Algebra, 1 (2011), 15.   Google Scholar

[1]

Hassan Mohammad. A diagonal PRP-type projection method for convex constrained nonlinear monotone equations. Journal of Industrial & Management Optimization, 2021, 17 (1) : 101-116. doi: 10.3934/jimo.2019101
[2]

C. J. Price. A modified Nelder-Mead barrier method for constrained optimization. Numerical Algebra, Control & Optimization, 2020  doi: 10.3934/naco.2020058
[3]

Liupeng Wang, Yunqing Huang. Error estimates for second-order SAV finite element method to phase field crystal model. Electronic Research Archive, 2021, 29 (1) : 1735-1752. doi: 10.3934/era.2020089
[4]

Jing Zhou, Cheng Lu, Ye Tian, Xiaoying Tang. A SOCP relaxation based branch-and-bound method for generalized trust-region subproblem. Journal of Industrial & Management Optimization, 2021, 17 (1) : 151-168. doi: 10.3934/jimo.2019104
[5]

Gang Luo, Qingzhi Yang. The point-wise convergence of shifted symmetric higher order power method. Journal of Industrial & Management Optimization, 2021, 17 (1) : 357-368. doi: 10.3934/jimo.2019115
[6]

Zuliang Lu, Fei Huang, Xiankui Wu, Lin Li, Shang Liu. Convergence and quasi-optimality of $ L^2- $norms based an adaptive finite element method for nonlinear optimal control problems. Electronic Research Archive, 2020, 28 (4) : 1459-1486. doi: 10.3934/era.2020077
[7]

Andrew D. Lewis. Erratum for "nonholonomic and constrained variational mechanics". Journal of Geometric Mechanics, 2020, 12 (4) : 671-675. doi: 10.3934/jgm.2020033
[8]

Leilei Wei, Yinnian He. A fully discrete local discontinuous Galerkin method with the generalized numerical flux to solve the tempered fractional reaction-diffusion equation. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020319
[9]

Yuxia Guo, Shaolong Peng. A direct method of moving planes for fully nonlinear nonlocal operators and applications. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020462
[10]

Xiu Ye, Shangyou Zhang, Peng Zhu. A weak Galerkin finite element method for nonlinear conservation laws. Electronic Research Archive, 2021, 29 (1) : 1897-1923. doi: 10.3934/era.2020097
[11]

Ke Su, Yumeng Lin, Chun Xu. A new adaptive method to nonlinear semi-infinite programming. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2021012
[12]

Li-Bin Liu, Ying Liang, Jian Zhang, Xiaobing Bao. A robust adaptive grid method for singularly perturbed Burger-Huxley equations. Electronic Research Archive, 2020, 28 (4) : 1439-1457. doi: 10.3934/era.2020076
[13]

Touria Karite, Ali Boutoulout. Global and regional constrained controllability for distributed parabolic linear systems: RHUM approach. Numerical Algebra, Control & Optimization, 2020  doi: 10.3934/naco.2020055
[14]

Chongyang Liu, Meijia Han, Zhaohua Gong, Kok Lay Teo. Robust parameter estimation for constrained time-delay systems with inexact measurements. Journal of Industrial & Management Optimization, 2021, 17 (1) : 317-337. doi: 10.3934/jimo.2019113
[15]

Yuan Tan, Qingyuan Cao, Lan Li, Tianshi Hu, Min Su. A chance-constrained stochastic model predictive control problem with disturbance feedback. Journal of Industrial & Management Optimization, 2021, 17 (1) : 67-79. doi: 10.3934/jimo.2019099
[16]

Helin Guo, Huan-Song Zhou. Properties of the minimizers for a constrained minimization problem arising in Kirchhoff equation. Discrete & Continuous Dynamical Systems - A, 2021, 41 (3) : 1023-1050. doi: 10.3934/dcds.2020308
[17]

Wenyan Zhuo, Honglin Yang, Leopoldo Eduardo Cárdenas-Barrón, Hong Wan. Loss-averse supply chain decisions with a capital constrained retailer. Journal of Industrial & Management Optimization, 2021, 17 (2) : 711-732. doi: 10.3934/jimo.2019131
[18]

Marion Darbas, Jérémy Heleine, Stephanie Lohrengel. Numerical resolution by the quasi-reversibility method of a data completion problem for Maxwell's equations. Inverse Problems & Imaging, 2020, 14 (6) : 1107-1133. doi: 10.3934/ipi.2020056
[19]

Alberto Bressan, Wen Shen. A posteriori error estimates for self-similar solutions to the Euler equations. Discrete & Continuous Dynamical Systems - A, 2021, 41 (1) : 113-130. doi: 10.3934/dcds.2020168
[20]

Mehdi Bastani, Davod Khojasteh Salkuyeh. On the GSOR iteration method for image restoration. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 27-43. doi: 10.3934/naco.2020013

2019 Impact Factor: 1.366

Tools

Metrics

  • PDF downloads (109)
  • HTML views (0)
  • Cited by (5)

Other articles
by authors

[Back to Top]

Copyright © 2020 American Institute of Mathematical Sciences