A conformal approach for surface inpainting

Lok Ming Lui; Chengfeng Wen; Xianfeng Gu

doi:10.3934/ipi.2013.7.863

Article Contents

2013, Volume 7, Issue 3: 863-884. Doi: 10.3934/ipi.2013.7.863

This issue Previous Article Video stabilization of atmospheric turbulence distortion Next Article Multi-view foreground segmentation via fourth order tensor learning

A conformal approach for surface inpainting

1.
Department of Mathematics, The Chinese University of Hong Kong, Shatin, New Territories, Hong Kong
2.
Department of Mathematics, The Chinese University of Hong Kong, Shatin, New Territories, Hong Kong,
3.
Department of Computer Sciences, Room 2425, Computer Science Building, State University of New York at Stony Brook, Stony Brook, New York 11794-4400

Received: November 30, 2012

Revised: March 31, 2013

Published: July 2013

Abstract Related Papers Cited by

Abstract

We address the problem of surface inpainting, which aims to fill in holes or missing regions on a Riemann surface based on its surface geometry. In practical situation, surfaces obtained from range scanners often have holes or missing regions where the 3D models are incomplete. In order to analyze the 3D shapes effectively, restoring the incomplete shape by filling in the surface holes is necessary. In this paper, we propose a novel conformal approach to inpaint surface holes on a Riemann surface based on its surface geometry. The basic idea is to represent the Riemann surface using its conformal factor and mean curvature. According to Riemann surface theory, a Riemann surface can be uniquely determined by its conformal factor and mean curvature up to a rigid motion. Given a Riemann surface $S$, its mean curvature $H$ and conformal factor $\lambda$ can be computed easily through its conformal parameterization. Conversely, given $\lambda$ and $H$, a Riemann surface can be uniquely reconstructed by solving the Gauss-Codazzi equation on the conformal parameter domain. Hence, the conformal factor and the mean curvature are two geometric quantities fully describing the surface. With this $\lambda$-$H$ representation of the surface, the problem of surface inpainting can be reduced to the problem of image inpainting of $\lambda$ and $H$ on the conformal parameter domain. The inpainting of $\lambda$ and $H$ can be done by conventional image inpainting models. Once $\lambda$ and $H$ are inpainted, a Riemann surface can be reconstructed which effectively restores the 3D surface with missing holes. Since the inpainting model is based on the geometric quantities $\lambda$ and $H$, the restored surface follows the surface geometric pattern as much as possible. We test the proposed algorithm on synthetic data, 3D human face data and MRI-derived brain surfaces. Experimental results show that our proposed method is an effective surface inpainting algorithm to fill in surface holes on an incomplete 3D models based their surface geometry.

Keywords:

Mathematics Subject Classification: Primary: 52C25, 49Q10, 65E05; Secondary: 65S05, 49M30.

Citation:

Related Papers

Cited by

References

[1]	T. F. Chan and J. Shen, Variational restoration of non-flat image features: Models and algorithms, SIAM Journal on Applied Mathematics, 61 (2001), 1338-1361.doi: 10.1137/S003613999935799X.
[2]	T. F. Chan and J. Shen, Mathematical models for local nontexture inpaintings, SIAM Journal on Applied Mathematics, 62 (2001/02), 1019-1043. doi: 10.1137/S0036139900368844.
[3]	S. Esedoglu and J. Shen, Digital inpainting based on the Mumford-Shah-Euler image model, Eureopean Journal on Applied Mathematics, 13 (2002), 353-370.doi: 10.1017/S0956792502004904.
[4]	T. F. Chan and J. Shen, Variational image inpainting, Communications in Pure and Applied Mathematics, 58 (2005), 579-619.doi: 10.1002/cpa.20075.
[5]	T. F. Chan, S. H. Kang and J. Shen, Euler's elastica and curvature-based inpainting, SIAM Journal on Applied Mathematics, 63 (2002), 564-592.doi: 10.1137/S0036139901390088.
[6]	S. Masnou and J. Morel, Level lines based disocclusion, $5^{th}$ IEEE International Conference on Image Processing, Chicago, IL, 3 (1998), 259-263.doi: 10.1109/ICIP.1998.999016.
[7]	M. Bertalmio, A. L. Bertozzi and G. Sapiro, Navier-Stokes, fluid dynamics, and image and video inpainting, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1 (2001), 355-362.doi: 10.1109/CVPR.2001.990497.
[8]	T. F. Chan, J. Shen and H.-M. Zhou, Total variation wavelet inpainting, Journal of Mathematical Imaging and Visions, 25 (2006), 107-125.doi: 10.1007/s10851-006-5257-3.
[9]	J. A. Dobrosotskaya and A. L. Bertozzi, A wavelet-Laplace variational technique for image deconvolution and inpainting, IEEE Transaction on Image Processing, 17 (2008), 657-663.doi: 10.1109/TIP.2008.919367.
[10]	A. Tsai, A. Yezzi, Jr. and A. S. Willsky, Curve evolution implementation of the Mumford-Shah functional for image segmentation, denoising, interpolation and magnification, IEEE Transaction on Image Processing, 10 (2001), 1169-1186.doi: 10.1109/83.935033.
[11]	M. Desbrun, M. Meyer, P. Schrder and A. H. Barr, Implicit fairing of irregular meshes using diffusion and curvature flow, in "Proceedings of the $26^{th}$ Annual Conference on Computer Graphics and Interactive Techniques," ACM Press/Addison-Wesley Publishing Co., New York, (1999), 317-324.doi: 10.1145/311535.311576.
[12]	G. Taubin, Geometric signal processing on polygonal meshes, Eurographics, (2000).
[13]	P. Smereka, Semi implicit level set methods curvature surface diffusion motion, Journal of Scientific Computing, 19 (2003), 439-456.doi: 10.1023/A:1025324613450.
[14]	M. Desbrun, M. Meyer, P. Schröder and A. H. Barr, Anisotropic feature preserving denoising of height fields and bivariate data, Graphics Interface, (2000).
[15]	M. Desbrun, M. Meyer, P. Schröder and A. H. Barr, Discrete differential geometry operators in nD, in "Proceedings on Vis. Math02," Springer-Verlag, Berlin, (2002), 35-57.
[16]	U. Clarenz, U. Diewald and M. Rumpf, Anisotropic diffusion in surface processing, in "Proceedings of IEEE Visualization 2000," (2000), 397-405.doi: 10.1109/VISUAL.2000.885721.
[17]	C. L. Bajaj and G. Xu, Anisotropic diffusion of subdivision surfaces and functions on surfaces, ACM Transaction on Graphics, 22 (2003), 4-32.
[18]	T. Tasdizen, R. Whitaker, P. Burchard and S. Osher, Geometric surface processing via anisotropic diffusion of normals, IEEE Visualization, (2002), 125-132.
[19]	T. Tasdizen, R. Whitaker, P. Burchard and S. Osher, Geometric surface processing via normal maps, ACM Transactions on Graphics, 22 (2003), 1012-1033.doi: 10.1145/944020.944024.
[20]	V. Caselles, G. Haro, G. Sapiro and J. Verdera, On geometric variational models for inpainting surface holes, Computer Vision and Image Understanding, 111 (2008), 351-373.doi: 10.1016/j.cviu.2008.01.002.
[21]	J. Verdera, V. Caselles, M. Bertalmio and G. Sapiro, Inpainting surface holes, in "Proceedings of International Conference on Image Processing, Vol. 2," (2003), 903-906.doi: 10.1109/ICIP.2003.1246828.
[22]	J. Davis, S. Marschner, M. Garr and M. Levoy, Filling holes in complex surfaces using volumetric diffusion, in "Proceedings of First International Symposium on 3D Data Processing, Visualization, and Transmission," (2002), 428-441.doi: 10.1109/TDPVT.2002.1024098.
[23]	U. Clarenz, U. Diewald, G. Dziuk, M. Rumpf and R. Rusu, A finite element method for surface restoration with smooth boundary conditions, Computer Aided Geometric Design, 21 (2004), 427-445.doi: 10.1016/j.cagd.2004.02.004.
[24]	A. Sharf, M. Alexa and D. Cohen, Context-based surface completion, ACM Transaction on Graphics, 23 (2004), 878-887.
[25]	V. Savchenko and N. Kojekine, An approach to blend surfaces, Advances in Modeling, Animation and Rendering, (2002), 139-150.doi: 10.1007/978-1-4471-0103-1_9.
[26]	M. K. Hurdal and K. Stephenson, Discrete conformal methods for cortical brain flattening, NeuroImage, 45 (2009), S86-S98.doi: 10.1016/j.neuroimage.2008.10.045.
[27]	B. Fischl, M. Sereno, R. Tootell and A. Dale, High-resolution intersubject averaging and a coordinate system for the cortical surface, Human Brain Mapping, 8 (1999), 272-284.
[28]	X. Gu, Y. Wang, T. F. Chan, P. M. Thompson and S.-T. Yau, Genus zero surface conformal mapping and its application to brain surface mapping, IEEE Transactions on Medical Imaging, 23 (2004), 949-958.
[29]	Y. Wang, L. M. Lui, X. Gu, K. M. Hayashi, T. F. Chan, A. W. Toga, P. M. Thompson and S.-T. Yau, Brain surface conformal parameterization using Riemann surface structure, IEEE Transactions on Medical Imaging, 26 (2007), 853-865.doi: 10.1109/TMI.2007.895464.
[30]	X. Gu and S.-T. Yau, Computing conformal structures of surfaces, Communication in Information System, 2 (2002), 121-145.
[31]	X. Gu, Y. Wang and S.-T. Yau, Geometric compression using Riemann surface structure, Communication in Information System, 3 (2004), 171-182.
[32]	L. M. Lui, T. W. Wong, W. Zeng, X. Gu, P. M. Thompson, T. F. Chan and S.-T. Yau, Optimization of surface registrations using Beltrami holomorphic flow, Journal of Scientific Computing, 50 (2012), 557-585.doi: 10.1007/s10915-011-9506-2.
[33]	L. M. Lui, T. W. Wong, X. Gu, P. M. Thompson, T. F. Chan and S.-T. Yau, Hippocampal shape registration using Beltrami holomorphic flow, Medical Image Computing and Computer Assisted Intervention(MICCAI), Part II, LNCS, 6362 (2010), 323-330.
[34]	L. M. Lui, K. C. Lam, T. W. Wong and X. Gu, Texture map and video compression using Beltrami representatio, to appear in SIAM Journal on Imaging Sciences, (2013).
[35]	S. Haker, S. Angenent, A. Tannenbaum, R. Kikinis, G. Sapiro and M. Halle, Conformal surface parameterization for texture mapping, IEEE Transaction of Visualization and Computer Graphics, 6 (2000), 181-189.
[36]	L. M. Lui, T. W. Wong, X. Gu, T. F. Chan and S.-T. Yau, Compression of surface diffeomorphism using Beltrami coefficient, IEEE Computer Vision and Pattern Recognition(CVPR), (2010), 2839-2846.
[37]	W. Zeng, L. M. Lui, F. Luo, T. F. Chan, S.-T. Yau and X. Gu, Computing quasiconformal maps using an auxiliary metric and discrete curvature flow, Numerische Mathematik, 121 (2012), 671-703.doi: 10.1007/s00211-012-0446-z.
[38]	L. M. Lui, K. C. Lam, S.-T. Yau and X. Gu, Teichmüller extremal mapping and its applications to landmark matching registration, arXiv:1211.2569.
[39]	W. Zeng and X. Gu, Registration for 3D surfaces with large deformations using quasi-conformal curvature flow, IEEE Conference on Computer Vision and Pattern Recognition (CVPR1), Jun 20-25, (2011), 2457-2464.
[40]	H. Zhao, S. Osher, B. Merriman and and M. Kang, Implicit and non-parametric shape reconstruction from unorganized points using variational level set method, Computer Vision and Image Understanding, 80 (2000), 295-319.
[41]	J. Liang, F. Park and H. Zhao, Robust and efficient implicit surface reconstruction of point clouds based on convexified image segmentation, Journal of Scientific Computing, 54 (2013), 577-602.doi: 10.1007/s10915-012-9674-8.
[42]	A. Wong and H. Zhao, Computation of quasiconformal surface maps using discrete Beltrami flow, UCLA CAM report 12-85.