American Institute of Mathematical Sciences

Advanced
  • Previous Article
    A phaseless inverse scattering problem for the 3-D Helmholtz equation
  • IPI Home
  • This Issue
  • Next Article
    Convergence of Tikhonov regularization for solving ill-posed operator equations with solutions defined on surfaces
April  2017, 11(2): 247-262. doi: 10.3934/ipi.2017012

A two-step mixed inpainting method with curvature-based anisotropy and spatial adaptivity

Francisco J. Ibarrola 1,2, and  Ruben D. Spies 3,4,5,,

1. 

Instituto de Investigación en Señales, Sistemas e Inteligencia Computacional, sinc(ⅰ), FICH-UNL/CONICET, Argentina
2. 

Ciudad Universitaria, CC 217, Ruta Nac. No 168, km 472.4, (3000) Santa Fe, Argentina
3. 

Instituto de Matemática Aplicada del Litoral, IMAL, CONICET-UNL, Centro Científico
4. 

Tecnológico CONICET Santa Fe, Colectora Ruta Nac. 168, km 472, Paraje "El Pozo", 3000, Santa Fe, Argentina and Departamento de Matemática
5. 

Facultad de Ingeniería Química, Universidad Nacional del Litoral, Santa Fe, Argentina

* Corresponding author: R. D. Spies

Received  December 2015 Revised  October 2016 Published  March 2017

The image inpainting problem consists of restoring an image from a (possibly noisy) observation, in which data from one or more regions are missing. Several inpainting models to perform this task have been developed, and although some of them perform reasonably well in certain types of images, quite a few issues are yet to be sorted out. For instance, if the image is expected to be smooth, the inpainting can be made with very good results by means of a Bayesian approach and a maximum a posteriori computation [2]. For non-smooth images, however, such an approach is far from being satisfactory. Even though the introduction of anisotropy by prior smooth gradient inpainting to the latter methodology is known to produce satisfactory results for slim missing regions [2], the quality of the restoration decays as the occluded regions widen. On the other hand, Total Variation (TV) inpainting models based on high order PDE diffusion equations can be used whenever edge restoration is a priority. More recently, the introduction of spatially variant conductivity coefficients on these models, such as in the case of Curvature-Driven Diffusion (CDD) [4], has allowed inpainted images with well defined edges and enhanced object connectivity. The CDD approach, nonetheless, is not quite suitable wherever the image is smooth, as it tends to produce piecewise constant restorations.

In this work we present a two-step inpainting process. The first step consists of using a CDD inpainting to build a pilot image from which to infer a-priori structural information on the image gradient. The second step is inpainting the image by minimizing a mixed spatially variant anisotropic functional, whose weight and penalization directions are based upon the aforementioned pilot image. Results are presented along with comparison measures in order to illustrate the performance of this inpainting method.

Keywords: Inpainting, inverse problems, ill-posedness, regularization, anisotropy.
Mathematics Subject Classification: Primary: 94A08, 68U10; Secondary: 65F22.
Citation: Francisco J. Ibarrola, Ruben D. Spies. A two-step mixed inpainting method with curvature-based anisotropy and spatial adaptivity. Inverse Problems & Imaging, 2017, 11 (2) : 247-262. doi: 10.3934/ipi.2017012
References:
[1]

R. Acar and C. R. Vogel, Analysis of bounded variation penalty methods for ill-posed problems, Inverse Problems, 10 (1994), 1217-1229.  doi: 10.1088/0266-5611/10/6/003.  Google Scholar

[2]

D. Calvetti, F. Sgallari and E. Somersalo, Image inpainting with structural boostrap priors, Image and Vision Computing, 24 (2006), 782-793.  doi: 10.1016/j.imavis.2006.01.015.  Google Scholar

[3]

T. Chan, S. Kang and J. Shen, Euler's elastica and curvature based inpaintings, SIAM J. on Applied Mathematics, 63 (2002), 564-592.  doi: 10.1137/S0036139901390088.  Google Scholar

[4]

T. Chan and J. Shen, Mathematical models for local nontexture inpaintings, SIAM J. on Applied Mathematics, 62 (2002), 1019-1043.  doi: 10.1137/S0036139900368844.  Google Scholar

[5]

F. Ibarrola and R. Spies, Image restoration with a half-quadratic approach to mixed weighted smooth and anisotropic bounded variation regularization, SOP Transactions on Applied Mathematics, 1 (2014), 57-95.   Google Scholar

[6]

J. Idier, Bayesian Approach to Inverse Problems, John Wiley & Sons, 2008. doi: 10.1002/9780470611197.  Google Scholar

[7]

F. Li, Z. Li and L. Pi, Variable exponent functionals in image restoration, Applied Mathematics and Computation, 216 (2010), 870-882.  doi: 10.1016/j.amc.2010.01.094.  Google Scholar

[8]

G. L. Mazzieri, R. D. Spies and K. G. Temperini, Mixed spatially varying $ L^2$-$ BV$ regularization of inverse ill-posed problems, Journal of Inverse and Ill-posed Problems, 23 (2015), 571-585.  doi: 10.1515/jiip-2014-0034.  Google Scholar

[9] R. Rockafellar, Convex Analysis, Princeton University Press, 1970.   Google Scholar
[10]

L. I. Rudin, S. Osher and E. Fatemi, Nonlinear total variation based noise removal algorithms (proceedings of the 11th annual international conference of the center for nonlinear studies), Physica D, 60 (1992), 259-268.  doi: 10.1016/0167-2789(92)90242-F.  Google Scholar

show all references

References:
[1]

R. Acar and C. R. Vogel, Analysis of bounded variation penalty methods for ill-posed problems, Inverse Problems, 10 (1994), 1217-1229.  doi: 10.1088/0266-5611/10/6/003.  Google Scholar

[2]

D. Calvetti, F. Sgallari and E. Somersalo, Image inpainting with structural boostrap priors, Image and Vision Computing, 24 (2006), 782-793.  doi: 10.1016/j.imavis.2006.01.015.  Google Scholar

[3]

T. Chan, S. Kang and J. Shen, Euler's elastica and curvature based inpaintings, SIAM J. on Applied Mathematics, 63 (2002), 564-592.  doi: 10.1137/S0036139901390088.  Google Scholar

[4]

T. Chan and J. Shen, Mathematical models for local nontexture inpaintings, SIAM J. on Applied Mathematics, 62 (2002), 1019-1043.  doi: 10.1137/S0036139900368844.  Google Scholar

[5]

F. Ibarrola and R. Spies, Image restoration with a half-quadratic approach to mixed weighted smooth and anisotropic bounded variation regularization, SOP Transactions on Applied Mathematics, 1 (2014), 57-95.   Google Scholar

[6]

J. Idier, Bayesian Approach to Inverse Problems, John Wiley & Sons, 2008. doi: 10.1002/9780470611197.  Google Scholar

[7]

F. Li, Z. Li and L. Pi, Variable exponent functionals in image restoration, Applied Mathematics and Computation, 216 (2010), 870-882.  doi: 10.1016/j.amc.2010.01.094.  Google Scholar

[8]

G. L. Mazzieri, R. D. Spies and K. G. Temperini, Mixed spatially varying $ L^2$-$ BV$ regularization of inverse ill-posed problems, Journal of Inverse and Ill-posed Problems, 23 (2015), 571-585.  doi: 10.1515/jiip-2014-0034.  Google Scholar

[9] R. Rockafellar, Convex Analysis, Princeton University Press, 1970.   Google Scholar
[10]

L. I. Rudin, S. Osher and E. Fatemi, Nonlinear total variation based noise removal algorithms (proceedings of the 11th annual international conference of the center for nonlinear studies), Physica D, 60 (1992), 259-268.  doi: 10.1016/0167-2789(92)90242-F.  Google Scholar

Figure 1.  (a) Occluded image; (b) T1 inpainting; (c) Anistoropic T1 inpainting
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 2.  Occluded image (a); TV inpainting (b)
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 3.  (a) Occluded image; (b) TV inpainting
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 4.  (a) Occluded image; (b) CDD inpainting
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 5.  (a) Occluded image; (b) Isotropic T1 inpainting; (c) CDD inpainting; (d) Mixed anisotropic T1-TV inpainting
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 6.  (a) Occluded noisy image; (b) Isotropic T1 inpainting; (c) CDD inpainting; (d) Anisotropic T1-TV inpainting
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 7.  (a) Occluded noisy image; (b) Isotropic T1 inpainting; (c) CDD inpainting; (d) Anisotropic T1-TV inpainting
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 8.  Left: CKS inpaintings; right: Anisotropic T1-TV inpaintings
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 9.  (a) Original image; (b) Occluded (masked) image; (c) T1 inpainting; (d) CDD inpainting; (e) T1-TV inpainting
Figure Options

Download full-size image

Download as PowerPoint slide

Table 1.  PSNR values for the test image (Figure 5)
Isotropic T1 CDD Anisotropic T1-TV
$\mathit{PSNR}$ 20.140 35.497 36.329
Table Options

Download as excel

Isotropic T1 CDD Anisotropic T1-TV
$\mathit{PSNR}$ 20.140 35.497 36.329
Table 2.  $\mathit{PSNR}$ values for grayscale image (Figure 6)
Isotropic T1 CDD Anisotropic T1-TV
$\mathit{PSNR}$ 29.127 29.952 30.868
Table Options

Download as excel

Isotropic T1 CDD Anisotropic T1-TV
$\mathit{PSNR}$ 29.127 29.952 30.868
Table 3.  $\mathit{PSNR}$ values for color image (Figure 7)
Isotropic T1 CDD Anisotropic T1-TV
$\mathit{PSNR}$ 20.568 21.360 22.112
Table Options

Download as excel

Isotropic T1 CDD Anisotropic T1-TV
$\mathit{PSNR}$ 20.568 21.360 22.112
Table 4.  $\mathit{PSNR}$ values for color image (Figure 7)
Gray CKS Gray A T1-TV Color CKS Color A T1-TV
$\mathit{PSNR}$ 29.770 32.421 21.464 24.284
Table Options

Download as excel

Gray CKS Gray A T1-TV Color CKS Color A T1-TV
$\mathit{PSNR}$ 29.770 32.421 21.464 24.284
[1]

Bernadette N. Hahn. Dynamic linear inverse problems with moderate movements of the object: Ill-posedness and regularization. Inverse Problems & Imaging, 2015, 9 (2) : 395-413. doi: 10.3934/ipi.2015.9.395
[2]

Adán J. Corcho. Ill-Posedness for the Benney system. Discrete & Continuous Dynamical Systems, 2006, 15 (3) : 965-972. doi: 10.3934/dcds.2006.15.965
[3]

Mahendra Panthee. On the ill-posedness result for the BBM equation. Discrete & Continuous Dynamical Systems, 2011, 30 (1) : 253-259. doi: 10.3934/dcds.2011.30.253
[4]

Xavier Carvajal, Mahendra Panthee. On ill-posedness for the generalized BBM equation. Discrete & Continuous Dynamical Systems, 2014, 34 (11) : 4565-4576. doi: 10.3934/dcds.2014.34.4565
[5]

Piero D'Ancona, Mamoru Okamoto. Blowup and ill-posedness results for a Dirac equation without gauge invariance. Evolution Equations & Control Theory, 2016, 5 (2) : 225-234. doi: 10.3934/eect.2016002
[6]

In-Jee Jeong, Benoit Pausader. Discrete Schrödinger equation and ill-posedness for the Euler equation. Discrete & Continuous Dynamical Systems, 2017, 37 (1) : 281-293. doi: 10.3934/dcds.2017012
[7]

Ye Zhang, Bernd Hofmann. Two new non-negativity preserving iterative regularization methods for ill-posed inverse problems. Inverse Problems & Imaging, 2021, 15 (2) : 229-256. doi: 10.3934/ipi.2020062
[8]

Xavier Carvajal, Liliana Esquivel, Raphael Santos. On local well-posedness and ill-posedness results for a coupled system of mkdv type equations. Discrete & Continuous Dynamical Systems, 2021, 41 (6) : 2699-2723. doi: 10.3934/dcds.2020382
[9]

Yonggeun Cho, Gyeongha Hwang, Soonsik Kwon, Sanghyuk Lee. Well-posedness and ill-posedness for the cubic fractional Schrödinger equations. Discrete & Continuous Dynamical Systems, 2015, 35 (7) : 2863-2880. doi: 10.3934/dcds.2015.35.2863
[10]

G. Fonseca, G. Rodríguez-Blanco, W. Sandoval. Well-posedness and ill-posedness results for the regularized Benjamin-Ono equation in weighted Sobolev spaces. Communications on Pure & Applied Analysis, 2015, 14 (4) : 1327-1341. doi: 10.3934/cpaa.2015.14.1327
[11]

Yannis Angelopoulos. Well-posedness and ill-posedness results for the Novikov-Veselov equation. Communications on Pure & Applied Analysis, 2016, 15 (3) : 727-760. doi: 10.3934/cpaa.2016.15.727
[12]

Shunlian Liu, David M. Ambrose. Sufficiently strong dispersion removes ill-posedness in truncated series models of water waves. Discrete & Continuous Dynamical Systems, 2019, 39 (6) : 3123-3147. doi: 10.3934/dcds.2019129
[13]

Jean-François Crouzet. 3D coded aperture imaging, ill-posedness and link with incomplete data radon transform. Inverse Problems & Imaging, 2011, 5 (2) : 341-353. doi: 10.3934/ipi.2011.5.341
[14]

Tsukasa Iwabuchi, Kota Uriya. Ill-posedness for the quadratic nonlinear Schrödinger equation with nonlinearity $|u|^2$. Communications on Pure & Applied Analysis, 2015, 14 (4) : 1395-1405. doi: 10.3934/cpaa.2015.14.1395
[15]

Matthew A. Fury. Regularization for ill-posed inhomogeneous evolution problems in a Hilbert space. Conference Publications, 2013, 2013 (special) : 259-272. doi: 10.3934/proc.2013.2013.259
[16]

Marcel Braukhoff. Semiconductor Boltzmann-Dirac-Benney equation with a BGK-type collision operator: Existence of solutions vs. ill-posedness. Kinetic & Related Models, 2019, 12 (2) : 445-482. doi: 10.3934/krm.2019019
[17]

Gabriel Peyré, Sébastien Bougleux, Laurent Cohen. Non-local regularization of inverse problems. Inverse Problems & Imaging, 2011, 5 (2) : 511-530. doi: 10.3934/ipi.2011.5.511
[18]

Philipp Hungerländer, Barbara Kaltenbacher, Franz Rendl. Regularization of inverse problems via box constrained minimization. Inverse Problems & Imaging, 2020, 14 (3) : 437-461. doi: 10.3934/ipi.2020021
[19]

Chao Deng, Xiaohua Yao. Well-posedness and ill-posedness for the 3D generalized Navier-Stokes equations in $\dot{F}^{-\alpha,r}_{\frac{3}{\alpha-1}}$. Discrete & Continuous Dynamical Systems, 2014, 34 (2) : 437-459. doi: 10.3934/dcds.2014.34.437
[20]

Felix Lucka, Katharina Proksch, Christoph Brune, Nicolai Bissantz, Martin Burger, Holger Dette, Frank Wübbeling. Risk estimators for choosing regularization parameters in ill-posed problems - properties and limitations. Inverse Problems & Imaging, 2018, 12 (5) : 1121-1155. doi: 10.3934/ipi.2018047

2020 Impact Factor: 1.639

Tools

Metrics

  • PDF downloads (127)
  • HTML views (54)
  • Cited by (1)

Other articles
by authors

[Back to Top]

Copyright © 2021 American Institute of Mathematical Sciences