Article Contents
Article Contents

# A two-step mixed inpainting method with curvature-based anisotropy and spatial adaptivity

• * Corresponding author: R. D. Spies
• The image inpainting problem consists of restoring an image from a (possibly noisy) observation, in which data from one or more regions are missing. Several inpainting models to perform this task have been developed, and although some of them perform reasonably well in certain types of images, quite a few issues are yet to be sorted out. For instance, if the image is expected to be smooth, the inpainting can be made with very good results by means of a Bayesian approach and a maximum a posteriori computation [2]. For non-smooth images, however, such an approach is far from being satisfactory. Even though the introduction of anisotropy by prior smooth gradient inpainting to the latter methodology is known to produce satisfactory results for slim missing regions [2], the quality of the restoration decays as the occluded regions widen. On the other hand, Total Variation (TV) inpainting models based on high order PDE diffusion equations can be used whenever edge restoration is a priority. More recently, the introduction of spatially variant conductivity coefficients on these models, such as in the case of Curvature-Driven Diffusion (CDD) [4], has allowed inpainted images with well defined edges and enhanced object connectivity. The CDD approach, nonetheless, is not quite suitable wherever the image is smooth, as it tends to produce piecewise constant restorations.

In this work we present a two-step inpainting process. The first step consists of using a CDD inpainting to build a pilot image from which to infer a-priori structural information on the image gradient. The second step is inpainting the image by minimizing a mixed spatially variant anisotropic functional, whose weight and penalization directions are based upon the aforementioned pilot image. Results are presented along with comparison measures in order to illustrate the performance of this inpainting method.

Mathematics Subject Classification: Primary: 94A08, 68U10; Secondary: 65F22.

 Citation:

• Figure 1.  (a) Occluded image; (b) T1 inpainting; (c) Anistoropic T1 inpainting

Figure 2.  Occluded image (a); TV inpainting (b)

Figure 3.  (a) Occluded image; (b) TV inpainting

Figure 4.  (a) Occluded image; (b) CDD inpainting

Figure 5.  (a) Occluded image; (b) Isotropic T1 inpainting; (c) CDD inpainting; (d) Mixed anisotropic T1-TV inpainting

Figure 6.  (a) Occluded noisy image; (b) Isotropic T1 inpainting; (c) CDD inpainting; (d) Anisotropic T1-TV inpainting

Figure 7.  (a) Occluded noisy image; (b) Isotropic T1 inpainting; (c) CDD inpainting; (d) Anisotropic T1-TV inpainting

Figure 8.  Left: CKS inpaintings; right: Anisotropic T1-TV inpaintings

Figure 9.  (a) Original image; (b) Occluded (masked) image; (c) T1 inpainting; (d) CDD inpainting; (e) T1-TV inpainting

Table 1.  PSNR values for the test image (Figure 5)

 Isotropic T1 CDD Anisotropic T1-TV $\mathit{PSNR}$ 20.140 35.497 36.329

Table 2.  $\mathit{PSNR}$ values for grayscale image (Figure 6)

 Isotropic T1 CDD Anisotropic T1-TV $\mathit{PSNR}$ 29.127 29.952 30.868

Table 3.  $\mathit{PSNR}$ values for color image (Figure 7)

 Isotropic T1 CDD Anisotropic T1-TV $\mathit{PSNR}$ 20.568 21.360 22.112

Table 4.  $\mathit{PSNR}$ values for color image (Figure 7)

 Gray CKS Gray A T1-TV Color CKS Color A T1-TV $\mathit{PSNR}$ 29.770 32.421 21.464 24.284
•  [1] R. Acar and C. R. Vogel, Analysis of bounded variation penalty methods for ill-posed problems, Inverse Problems, 10 (1994), 1217-1229.  doi: 10.1088/0266-5611/10/6/003. [2] D. Calvetti, F. Sgallari and E. Somersalo, Image inpainting with structural boostrap priors, Image and Vision Computing, 24 (2006), 782-793.  doi: 10.1016/j.imavis.2006.01.015. [3] T. Chan, S. Kang and J. Shen, Euler's elastica and curvature based inpaintings, SIAM J. on Applied Mathematics, 63 (2002), 564-592.  doi: 10.1137/S0036139901390088. [4] T. Chan and J. Shen, Mathematical models for local nontexture inpaintings, SIAM J. on Applied Mathematics, 62 (2002), 1019-1043.  doi: 10.1137/S0036139900368844. [5] F. Ibarrola and R. Spies, Image restoration with a half-quadratic approach to mixed weighted smooth and anisotropic bounded variation regularization, SOP Transactions on Applied Mathematics, 1 (2014), 57-95. [6] J. Idier, Bayesian Approach to Inverse Problems, John Wiley & Sons, 2008. doi: 10.1002/9780470611197. [7] F. Li, Z. Li and L. Pi, Variable exponent functionals in image restoration, Applied Mathematics and Computation, 216 (2010), 870-882.  doi: 10.1016/j.amc.2010.01.094. [8] G. L. Mazzieri, R. D. Spies and K. G. Temperini, Mixed spatially varying $L^2$-$BV$ regularization of inverse ill-posed problems, Journal of Inverse and Ill-posed Problems, 23 (2015), 571-585.  doi: 10.1515/jiip-2014-0034. [9] R. Rockafellar,  Convex Analysis, Princeton University Press, 1970. [10] L. I. Rudin, S. Osher and E. Fatemi, Nonlinear total variation based noise removal algorithms (proceedings of the 11th annual international conference of the center for nonlinear studies), Physica D, 60 (1992), 259-268.  doi: 10.1016/0167-2789(92)90242-F.

Figures(9)

Tables(4)