Augmented Lagrangian method for total variation restoration with non-quadratic fidelity

Chunlin Wu; Juyong Zhang; Xue-Cheng Tai

doi:10.3934/ipi.2011.5.237

Article Contents

2011, Volume 5, Issue 1: 237-261. Doi: 10.3934/ipi.2011.5.237

This issue Previous Article Structural stability in a minimization problem and applications to conductivity imaging Next Article An inviscid model for nonrigid image registration

Augmented Lagrangian method for total variation restoration with non-quadratic fidelity

1.
Division of Mathematical Sciences, School of Physical & Mathematical Sciences, Nanyang Technological University
2.
Division of Computer Communications, School of Computer Engineering, Nanyang Technological University
3.
University of Bergen, University of Bergen Bergen

Received: November 30, 2009

Revised: August 31, 2010

Published: January 2011

Abstract Related Papers Cited by

Abstract

Recently augmented Lagrangian method has been successfully applied to image restoration. We extend the method to total variation (TV) restoration models with non-quadratic fidelities. We will first introduce the method and present an iterative algorithm for TV restoration with a quite general fidelity. In each iteration, three sub-problems need to be solved, two of which can be very efficiently solved via Fast Fourier Transform (FFT) implementation or closed form solution. In general the third sub-problem need iterative solvers. We then apply our method to TV restoration with $L^1$ and Kullback-Leibler (KL) fidelities, two common and important data terms for deblurring images corrupted by impulsive noise and Poisson noise, respectively. For these typical fidelities, we show that the third sub-problem also has closed form solution and thus can be efficiently solved. In addition, convergence analysis of these algorithms are given. Numerical experiments demonstrate the efficiency of our method.

Keywords:

Mathematics Subject Classification: 80M30, 80M50, 68U10.

Citation:

Related Papers

Cited by

References

[1]	S. Alliney, Digital filters as absolute norm regularizers, IEEE Trans. Signal Process., 40 (1992), 1548-1562.doi: 10.1109/78.139258.
[2]	P. Besbeas, I. D. Fies and T. Sapatinas, A comparative simulation study of wavelet shrinkage estimators for Poisson counts, International Statistical Review, 72 (2004), 209-237.doi: 10.1111/j.1751-5823.2004.tb00234.x.
[3]	P. Blomgren and T. F. Chan, Color TV: Total variation methods for restoration of vector-valued images, IEEE Trans. Image Process., 7 (1998), 304-309.doi: 10.1109/83.661180.
[4]	A. Bovik, "Handbook of Image and Video Processing," Academic Press, 2000.
[5]	X. Bresson and T. F. Chan, Fast dual minimization of the vectorial total variation norm and applications to color image processing, Inverse Problems and Imaging, 2 (2008), 455-484.doi: 10.3934/ipi.2008.2.455.
[6]	C. Brune, A. Sawatzky and M. Burger, Bregman-EM-TV methods with application to optical nanoscopy, LNCS, 5567 (2009), 235-246.
[7]	A. Caboussat, R. Glowinski and V. Pons, An augmented Lagrangian approach to the numerical solution of a non-smooth eigenvalue problem, J. Numer. Math., 17 (2009), 3-26.doi: 10.1515/JNUM.2009.002.
[8]	E. Candes, J. Romberg and T. Tao, Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information, IEEE Trans. Inform. Theory, 52 (2006), 489-509.doi: 10.1109/TIT.2005.862083.
[9]	J. L. Carter, "Dual Methods for Total Variation - Based Image Restoration," Ph.D thesis, UCLA, 2001.
[10]	A. Chambolle and P. L. Lions, Image recovery via total variation minimization and related problems, Numer. Math., 76 (1997), 167-188.doi: 10.1007/s002110050258.
[11]	A. Chambolle, An algorithm for total variation minimization and applications, J. Math. Imaging Vision, 20 (2004), 89-97.doi: 10.1023/B:JMIV.0000011321.19549.88.
[12]	R. Chan, C. W. Ho and M. Nikolova, Salt-and-pepper noise removal by median-type noise detector and detail-preserving regularization, IEEE Trans. Image Process., 14 (2005), 1479-1485.doi: 10.1109/TIP.2005.852196.
[13]	R. H. Chan and K. Chen, Multilevel algorithms for a Poisson noise removal model with total variation regularization, Int. J. Comput. Math., 84 (2007), 1183-1198.doi: 10.1080/00207160701450390.
[14]	T. F. Chan, G. H. Golub and P. Mulet, A nonlinear primal-dual method for total variation-based image restoration, SIAM J. Sci. Comput., 20 (1999), 1964-1977.doi: 10.1137/S1064827596299767.
[15]	T. Chan, A. Marquina and P. Mulet, High-order total variation-based image restoration, SIAM J. Sci. Comput., 22 (2000), 503-516.doi: 10.1137/S1064827598344169.
[16]	T. F. Chan, S. H. Kang and J. H. Shen, Total variation denoising and enhancement of color images based on the CB and HSV color models, J. Visual Commun. Image Repres., 12 (2001), 422-435.doi: 10.1006/jvci.2001.0491.
[17]	T. F. Chan and S. Esedoglu, Aspects of total variation regularized $L^1$ function approximation, SIAM J. Appl. Math., 65 (2005), 1817-1837.doi: 10.1137/040604297.
[18]	S. Chen, D. Donoho and M. A. Saunders, Atomic decomposition by basis pursuit, SIAM J. Sci. Comput., 20 (1998), pp. 33-61.doi: 10.1137/S1064827596304010.
[19]	T. Chen and H. R. Wu, Space variant median filters for the restoration of impulse noise corrupted images, IEEE Trans. Circuits Syst. II, Analog Digit. Signal Process., 48 (2001), 784-789.
[20]	Y. Dong, M. Hintermüller and M. Neri, An efficient primal-dual method for $L^1$TV image restoration, SIAM J. Imaging Sciences, 2 (2009), 1168-1189.doi: 10.1137/090758490.
[21]	D. L. Donoho, Compressed sensing, IEEE Trans. Inform. Theory, 52 (2006), 1289-1306.doi: 10.1109/TIT.2006.871582.
[22]	I. Ekeland and R. Témam, "Convex Analysis and Variational Problems," SIAM, 1999.
[23]	H. L. Eng and K. K. Ma, Noise adaptive soft-switching median filter, IEEE Trans. Image Process., 10 (2001), 242-251.doi: 10.1109/83.902289.
[24]	E. Esser, Applications of Lagrangian-based alternating direction methods and connections to split Bregman, UCLA CAM Report, cam09-31. Available from: ftp://ftp.math.ucla.edu/pub/camreport/cam09-31.pdf.
[25]	M. A. T. Figueiredo and J. M. Bioucas-Dias, Deconvolution of Poissonian images using variable splitting and augmented Lagrangian optimization, in "IEEE Workshop on Statistical Signal Processing," Cardiff, (2009), 733-736.doi: 10.1109/SSP.2009.5278459.
[26]	H. Y. Fu, M. K. Ng, M. Nikolova and J. L. Barlow, Efficient minimization methods of mixed l2-l1 and l1-l1 norms for image restoration, SIAM J. Sci. Comput., 27 (2006), 1881-1902.doi: 10.1137/040615079.
[27]	R. Glowinski and P. Le Tallec, "Augmented Lagrangians and Operator-Splitting Methods in Nonlinear Mechanics," SIAM, Philadelphia, 1989.
[28]	T. Goldstein and S. Osher, The split Bregman method for L1 regularized problems, SIAM J. Imaging Sciences, 2 (2009), 323-343.doi: 10.1137/080725891.
[29]	M. R. Hestenes, Multiplier and gradient methods, J. Optim. Theory and Appl., 4 (1969), 303-320.doi: 10.1007/BF00927673.
[30]	W. Hinterberger and O. Scherzer, Variational methods on the space of functions of bounded Hessian for convexification and denoising, Computing, 76 (2006), 109-133.doi: 10.1007/s00607-005-0119-1.
[31]	M. Hintermüller, K. Ito and K. Kunisch, The primal-dual active set strategy as a semismooth Newton method, SIAM J. Optim., 13 (2002), 865-888.doi: 10.1137/S1052623401383558.
[32]	Y. Huang, M. Ng and Y. Wen, A fast total variation minimization method for image restoration, SIAM Multi. Model. Simul., 7 (2008), 774-795.doi: 10.1137/070703533.
[33]	H. Hwang and R. A. Haddad, Adaptive median filters: New algorithms and results, IEEE Trans. Image Process., 4 (1995), 499-502.doi: 10.1109/83.370679.
[34]	C. Kervrann and A. Trubuil, An adaptive window approach for poisson noise reduction and structure preserving in confocal microscopy, in "Proc. International Symposium on Biomedical Imaging (ISBI'04)," Arlington, (2004), 788-791.
[35]	E. Kolaczyk, Wavelet shrinkage estimation of certain Poisson intensity signals using corrected thresholds, Statist. Sinica, 9 (1999), 119-135.
[36]	T. Le, R. Chartrand and T. J. Asaki, A variational approach to reconstructing images corrupted by Poisson noise, J. Math. Imaging Vision, 27 (2007), 257-263.doi: 10.1007/s10851-007-0652-y.
[37]	Y. Li and S. Osher, A new median formula with applications to PDE based denoising, Commun. Math. Sci., 7 (2009), 741-753.
[38]	M. Lysaker, A. Lundervold and X.-C. Tai, Noise removal using fourth-order partial differential equation with applications to medical magnetic resonance images in space and time, IEEE Trans. Image Process., 12 (2003), 1579-1590.doi: 10.1109/TIP.2003.819229.
[39]	M. Lysaker and X.-C. Tai, Iterative image restoration combining total variation minimization and a second order functional, Int'l J. Computer Vision, 2005.
[40]	P. Mrazek, J. Weickert and A. Bruhn, On robust estimations and smoothing with spatial and tonal kernels, in "Geometric Properties from Incomplete Data," Springer, (2006), 335-352.doi: 10.1007/1-4020-3858-8_18.
[41]	P. E. Ng and K. K. Ma, A switching median filter with boundary discriminative noise detection for extremely corrupted images, IEEE Trans. Image Process., 15 (2006), 1506-1516.doi: 10.1109/TIP.2005.871129.
[42]	M. Nikolova, Minimizers of cost-functions involving non-smooth data fidelity terms, SIAM J. Num. Anal., 40 (2002), 965-994.doi: 10.1137/S0036142901389165.
[43]	M. Nikolova, A variational approach to remove outliers and impulse noise, J. Math. Imaging Vision, 20 (2004), 99-120.doi: 10.1023/B:JMIV.0000011920.58935.9c.
[44]	V. Y. Panin, G. L. Zeng and G. T. Gullberg, Total variation regulated EM algorithm [SPECT reconstruction], IEEE Trans. Nucl. Sci., 46 (1999), 2202-2210.doi: 10.1109/23.819305.
[45]	G. Pok, J. C. Liu and A. S. Nair, Selective removal of impulse noise based on homogeneity level information, IEEE Trans. Image Process., 12 (2003), 85-92.doi: 10.1109/TIP.2002.804278.
[46]	M. J. D. Powell, A method for nonlinear constraints in minimization problems, in "Optimization"(ed. R. Fletcher), Academic Press, (1972), 283-298.
[47]	R. T. Rockafellar, A dual approach to solving nonlinear programming problems by unconstrained optimization, Mathematical Programming, 5 (1973), 354-373.doi: 10.1007/BF01580138.
[48]	L. Rudin, S. Osher and E. Fatemi, Nonlinear total variation based noise removal algorithms, Physica D, 60 (1992), 259-268.doi: 10.1016/0167-2789(92)90242-F.
[49]	G. Sapiro and D. L. Ringach, Anisotropic diffusion of multivalued images with applications to color filtering, IEEE Trans. Image Process., 5 (1996), 1582-1586.doi: 10.1109/83.541429.
[50]	O. Scherer, Denoising with higher order derivatives of bounded variation and an application to parameter estimation, Computing, 60 (1998), 1-27.doi: 10.1007/BF02684327.
[51]	S. Setzer, Split Bregman algorithm, Douglas-Rachford splitting and frame shrinkage, LNCS, 5567 (2009), 464-476.
[52]	S. Setzer, G. Steidl and T. Teuber, Deblurring Poissonian images by split Bregman techniques, J. Visual Commun. Image Repres., accepted (2009).
[53]	L. A. Shepp and Y. Vardi, Maximum likelihood reconstruction for emission tomography, IEEE Trans. Medical Imaging, 1 (1982), 113-122.doi: 10.1109/TMI.1982.4307558.
[54]	X. C. Tai and C. L. Wu, Augmented Lagrangian method, dual methods and split Bregman iteration for ROF model, LNCS, 5567 (2009), 502-513.
[55]	K. Timmermann and R. Novak, Multiscale modeling and estimation of Poisson processes with applications to photon-limited imaging, IEEE Trans. Inf. Theor., 45 (1999), 846-852.doi: 10.1109/18.761328.
[56]	Y. Wang, J. Yang, W. Yin and Y. Zhang, A new alternating minimization algorithm for total variation image reconstruction, SIAM J. Imaging Sciences, 1 (2008), 248-272.doi: 10.1137/080724265.
[57]	C. L. Wu and X. C. Tai, Augmented Lagrangian method, dual methods, and split Bregman iteration for ROF, vectorial TV, and high order models, SIAM J. Imaging Sciences, 3 (2010), 300-339.doi: 10.1137/090767558.
[58]	J. F. Yang, W. T. Yin, Y. Zhang and Y. L. Wang, A fast algorithm for edge-preserving variational multichannel image restoration, SIAM J. Imaging Sciences, 2 (2009), 569-592.doi: 10.1137/080730421.
[59]	J. Yang, Y. Zhang and W. Yin, An efficient TVL1 algorithm for deblurring multichannel images corrupted by impulsive noise, SIAM J. Sci. Comput., 31 (2009), 2842-2865.doi: 10.1137/080732894.
[60]	W. Yin, D. Goldfarb and S. Osher, Image cartoon-texture decomposition and feature selection using the total variation regularized $L^1$ functional, LNCS, 3752 (2005), 73-84.
[61]	W. Yin, D. Goldfarb and S. Osher, The total variation regularized $L^1$ model for multiscale decomposition, Multi. Model. Simul., 6 (2007), 190-211.doi: 10.1137/060663027.
[62]	W. T. Yin, S. Osher, D. Goldfarb and J. Darbon, Bregman iterative algorithms for compressend sensing and related problems, SIAM J. Imaging Sciences, 1 (2008), 143-168.doi: 10.1137/070703983.
[63]	W. T. Yin, Analysis and generalizations of the linearized Bregman method, SIAM J. Imaging Sciences, 3 (2010), 856-877.doi: 10.1137/090760350.
[64]	Y.-L. You and M. Kaveh, Fourth-order partial differential equation for noise removal, IEEE Trans. Image Process., 9 (2000), 1723-1730.doi: 10.1109/83.869184.
[65]	R. Zanella, P. Boccacci, L. Zanni and M. Bertero, Efficient gradient projection methods for edge-preserving removal of Poisson noise, Inverse Problems, 25 (2009).
[66]	M. Zhu and T. F. Chan, An efficient primal-dual hybrid gradient algorithm for total variation image restoration, UCLA CAM Report, cam08-34. Available from: ftp://ftp.math.ucla.edu/pub/camreport/cam08-34.pdf.
[67]	M. Zhu, S. J. Wright and T. F. Chan, Duality-based algorithms for total variation image restoration, Comput. Optim. Appl., (2008). Available from: http://www.springerlink.com/content/l2k84n130x6l4332/.