March  2021, 14(3): 971-985. doi: 10.3934/dcdss.2020351

3D image segmentation supported by a point cloud

1. 

Department of Mathematics and Descriptive Geometry, Faculty of Civil Engineering, Slovak University of Technology, Radlinského 11, 810 05 Bratislava, Slovakia

2. 

Department of Mathematics University of Nigeria 410001 Nsukka, Nigeria

3. 

Mammalian Embryo and Stem Cell Group, University of Cambridge, Department of Physiology, Development and Neuroscience, Downing Street, Cambridge CB2 3EG, UK

4. 

Department of Biological Sciences University of Cyprus University Avenue 1, Nicosia 2109, Cyprus

Received  January 2019 Revised  October 2019 Published  May 2020

Here, we report a novel method of 3D image segmentation, using surface reconstruction from 3D point cloud data and 3D digital image information. For this task, we apply a mathematical model and numerical method based on the level set algorithm. This method solves surface reconstruction by the application of advection equation with a curvature term, which gives the evolution of an initial condition to the final state. This is done by defining the advective velocity in the level set equation as the weighted sum of distance function and edge detector function gradients. The distance function to the shape, represented by the point cloud, is computed using the fast sweeping method. The edge detector function is applied to the presmoothed 3D image. A crucial point for efficiency is the construction of an initial condition by a simple tagging algorithm, which allows us also to highly speed up the numerical scheme when solving PDEs. For the numerical discretization, we use a semi-implicit co-volume scheme in the curvature part and implicit upwind scheme in the advective part. The method was tested on representative examples and applied to real data representing 3D biological microscopic images of developing mammalian embryo.

Citation: Balázs Kósa, Karol Mikula, Markjoe Olunna Uba, Antonia Weberling, Neophytos Christodoulou, Magdalena Zernicka-Goetz. 3D image segmentation supported by a point cloud. Discrete & Continuous Dynamical Systems - S, 2021, 14 (3) : 971-985. doi: 10.3934/dcdss.2020351
References:
[1]

N. ChristodoulouC. KyprianouA. WeberlingR. Wang and G. Cui, Sequential formation and resolution of multiple rosettes drive embryo remodeling after implantation, Nature Cell Biology, 20 (2018), 1278-1289.  doi: 10.1038/s41556-018-0211-3.  Google Scholar

[2]

S. CorsaroK. MikulaA. Sarti and F. Sgallari, Semi-implicit covolume method in 3D image segmentation, SIAM J. Sci. Comput., 28 (2006), 2248-2265.  doi: 10.1137/060651203.  Google Scholar

[3]

S. Dyballa, T. Savy, P. Germann, K. Mikula and M.Remešíková, et al., Distribution of neurosensory progenitor pools during inner ear morphogenesis unveiled by cell lineage reconstruction, 6 (2017). doi: 10.7554/eLife.22268.001.  Google Scholar

[4]

L. C. Evans and J. Spruck, Motion of level sets by mean curvature. Ⅰ, J. Differential Geom., 33 (1991), 635-681.  doi: 10.4310/jdg/1214446559.  Google Scholar

[5]

R. EymardA. Handlovičvá and K. Mikula, Study of a finite volume scheme for the regularised mean curvature flow level set equation, IMA J. Numer. Anal., 31 (2011), 813-846.  doi: 10.1093/imanum/drq025.  Google Scholar

[6]

E. Faure, et al., A workflow to process 3D+time microscopy images of developing organisms and reconstruct their cell lineage, Nature Communications, 7 (2016). doi: 10.1038/ncomms9674.  Google Scholar

[7]

B. Kósa, J. Haličková-Brehovská and K. Mikula, New efficient numerical method for 3D point cloud surface reconstruction by using level set methods, Proceedings of Equadiff 2017 Conference, 2017,387–396. Google Scholar

[8]

K. Mikula, N. Peyriéras, M. Remešíková and A. Sarti, 3D embryogenesis image segmentation by the generalized subjective surface method using the finite volume technique, in Finite Volumes for Complex Applications V, ISTE, London, 2008,585-592.  Google Scholar

[9]

K. Mikula and M. Remešíková, Finite volume schemes for the generalized subjective surface equation in image segmentation, Kybernetika, 45 (2009), 646-656.   Google Scholar

[10]

K. Mikula and A. Sarti, Parallel co-volume subjective surface method for 3D medical image segmentation, in Deformable Models, Topics in Biomedical Engineering. International Book Series, Springer, NY, 2007,123–160. doi: 10.1007/978-0-387-68343-0_5.  Google Scholar

[11]

S. Osher and R. Fedkiw, Level Set Methods and Dynamic Implicit Surfaces, Applied Mathematical Sciences, 153, Springer-Verlag, New York, 2003. doi: 10.1007/b98879.  Google Scholar

[12]

P. Perona and J. Malik, Scale-space and edge detection using anisotropic diffusion, IEEE Trans. Pattern Anal. Machine Intelligence, 12 (1990), 629-639.  doi: 10.1109/34.56205.  Google Scholar

[13]

A. SartiR. Malladi and J. A. Sethian, Subjective surfaces: A method for completing missing boundaries, Proc. Natl. Acad. Sci. USA, 97 (2000), 6258-6263.  doi: 10.1073/pnas.110135797.  Google Scholar

[14] J. A. Sethian, Level Set Methods and Fast Marching Methods: Evolving Interfaces in Computational Geometry, Fluid Mechanics, Computer Vision, and Material Science, Cambridge Monographs on Applied and Computational Mathematics, 3, Cambridge University Press, Cambridge, 1999.   Google Scholar
[15]

M. N. ShahbaziA. ScialdoneN. SkorupskaA. Weberling and G. Recher, Pluripotent state transitions coordinate morphogenesis in mouse and human embryos, Nature, 552 (2017), 239-243.  doi: 10.1038/nature24675.  Google Scholar

[16]

M. N. Shahbazi and M. Zernicka-Goetz, Deconstructing and reconstructing the mouse and human early embryo, Nature Cell Biology, 20 (2018), 878-887.  doi: 10.1038/s41556-018-0144-x.  Google Scholar

[17]

C. ZanellaM. CampanaB. RizziC. Melani and G. Sanguinetti, Cells segmentation from 3-D confocal images of early zebrafish embryogenesis, IEEE Trans. Image Process., 19 (2010), 770-781.  doi: 10.1109/TIP.2009.2033629.  Google Scholar

[18]

H. Zhao, A fast sweeping method for Eikonal equations, Math. Comp., 74 (2005), 603-627.  doi: 10.1090/S0025-5718-04-01678-3.  Google Scholar

[19]

H. ZhaoS. OsherB. Merriman and M. Kang, Implicit and nonparametric shape reconstruction from unorganized data using a variational level set method, Comput. Vision Image Understanding, 80 (2000), 295-319.  doi: 10.1006/cviu.2000.0875.  Google Scholar

show all references

References:
[1]

N. ChristodoulouC. KyprianouA. WeberlingR. Wang and G. Cui, Sequential formation and resolution of multiple rosettes drive embryo remodeling after implantation, Nature Cell Biology, 20 (2018), 1278-1289.  doi: 10.1038/s41556-018-0211-3.  Google Scholar

[2]

S. CorsaroK. MikulaA. Sarti and F. Sgallari, Semi-implicit covolume method in 3D image segmentation, SIAM J. Sci. Comput., 28 (2006), 2248-2265.  doi: 10.1137/060651203.  Google Scholar

[3]

S. Dyballa, T. Savy, P. Germann, K. Mikula and M.Remešíková, et al., Distribution of neurosensory progenitor pools during inner ear morphogenesis unveiled by cell lineage reconstruction, 6 (2017). doi: 10.7554/eLife.22268.001.  Google Scholar

[4]

L. C. Evans and J. Spruck, Motion of level sets by mean curvature. Ⅰ, J. Differential Geom., 33 (1991), 635-681.  doi: 10.4310/jdg/1214446559.  Google Scholar

[5]

R. EymardA. Handlovičvá and K. Mikula, Study of a finite volume scheme for the regularised mean curvature flow level set equation, IMA J. Numer. Anal., 31 (2011), 813-846.  doi: 10.1093/imanum/drq025.  Google Scholar

[6]

E. Faure, et al., A workflow to process 3D+time microscopy images of developing organisms and reconstruct their cell lineage, Nature Communications, 7 (2016). doi: 10.1038/ncomms9674.  Google Scholar

[7]

B. Kósa, J. Haličková-Brehovská and K. Mikula, New efficient numerical method for 3D point cloud surface reconstruction by using level set methods, Proceedings of Equadiff 2017 Conference, 2017,387–396. Google Scholar

[8]

K. Mikula, N. Peyriéras, M. Remešíková and A. Sarti, 3D embryogenesis image segmentation by the generalized subjective surface method using the finite volume technique, in Finite Volumes for Complex Applications V, ISTE, London, 2008,585-592.  Google Scholar

[9]

K. Mikula and M. Remešíková, Finite volume schemes for the generalized subjective surface equation in image segmentation, Kybernetika, 45 (2009), 646-656.   Google Scholar

[10]

K. Mikula and A. Sarti, Parallel co-volume subjective surface method for 3D medical image segmentation, in Deformable Models, Topics in Biomedical Engineering. International Book Series, Springer, NY, 2007,123–160. doi: 10.1007/978-0-387-68343-0_5.  Google Scholar

[11]

S. Osher and R. Fedkiw, Level Set Methods and Dynamic Implicit Surfaces, Applied Mathematical Sciences, 153, Springer-Verlag, New York, 2003. doi: 10.1007/b98879.  Google Scholar

[12]

P. Perona and J. Malik, Scale-space and edge detection using anisotropic diffusion, IEEE Trans. Pattern Anal. Machine Intelligence, 12 (1990), 629-639.  doi: 10.1109/34.56205.  Google Scholar

[13]

A. SartiR. Malladi and J. A. Sethian, Subjective surfaces: A method for completing missing boundaries, Proc. Natl. Acad. Sci. USA, 97 (2000), 6258-6263.  doi: 10.1073/pnas.110135797.  Google Scholar

[14] J. A. Sethian, Level Set Methods and Fast Marching Methods: Evolving Interfaces in Computational Geometry, Fluid Mechanics, Computer Vision, and Material Science, Cambridge Monographs on Applied and Computational Mathematics, 3, Cambridge University Press, Cambridge, 1999.   Google Scholar
[15]

M. N. ShahbaziA. ScialdoneN. SkorupskaA. Weberling and G. Recher, Pluripotent state transitions coordinate morphogenesis in mouse and human embryos, Nature, 552 (2017), 239-243.  doi: 10.1038/nature24675.  Google Scholar

[16]

M. N. Shahbazi and M. Zernicka-Goetz, Deconstructing and reconstructing the mouse and human early embryo, Nature Cell Biology, 20 (2018), 878-887.  doi: 10.1038/s41556-018-0144-x.  Google Scholar

[17]

C. ZanellaM. CampanaB. RizziC. Melani and G. Sanguinetti, Cells segmentation from 3-D confocal images of early zebrafish embryogenesis, IEEE Trans. Image Process., 19 (2010), 770-781.  doi: 10.1109/TIP.2009.2033629.  Google Scholar

[18]

H. Zhao, A fast sweeping method for Eikonal equations, Math. Comp., 74 (2005), 603-627.  doi: 10.1090/S0025-5718-04-01678-3.  Google Scholar

[19]

H. ZhaoS. OsherB. Merriman and M. Kang, Implicit and nonparametric shape reconstruction from unorganized data using a variational level set method, Comput. Vision Image Understanding, 80 (2000), 295-319.  doi: 10.1006/cviu.2000.0875.  Google Scholar

Figure 1.  The voxel grid cell with a tetrahedral finite element
Figure 2.  Notation for the additional points of a grid cell used for calculation of $ Gu_{i,j,k}^{n,l},\;l = 1,...,24 $
Figure 3.  Tetrahedral finite element with marked edges for approximation of partial derivatives
Figure 4.  In the first column there are 3D images of a sphere with 6 symmetrically placed holes. The experiment was executed on spheres with holes of different shapes and sizes. The second column shows the result of the mathematical model (1) with $ \theta = 0 $ and $ \rho = 1 $ for the advective velocity (3). In the third column we show the result after we added points to the missing parts of the sphere and set $ \theta = 1 $, $ \rho = 1 $ in (3)
Figure 5.  Sections of the embryo together with the marked points for visceral endoderm (VE)
Figure 6.  3D image of embryo structure, visualized with the reconstructed ExE part of the embryo in the upper picture and with VE part in the lower one
Figure 7.  Reconstructed surface of extraembryonic ectoderm (ExE) and visceral endoderm (VE), visualized with point cloud
[1]

Sandrine Anthoine, Jean-François Aujol, Yannick Boursier, Clothilde Mélot. Some proximal methods for Poisson intensity CBCT and PET. Inverse Problems & Imaging, 2012, 6 (4) : 565-598. doi: 10.3934/ipi.2012.6.565

[2]

Habib Ammari, Josselin Garnier, Vincent Jugnon. Detection, reconstruction, and characterization algorithms from noisy data in multistatic wave imaging. Discrete & Continuous Dynamical Systems - S, 2015, 8 (3) : 389-417. doi: 10.3934/dcdss.2015.8.389

[3]

Lekbir Afraites, Abdelghafour Atlas, Fahd Karami, Driss Meskine. Some class of parabolic systems applied to image processing. Discrete & Continuous Dynamical Systems - B, 2016, 21 (6) : 1671-1687. doi: 10.3934/dcdsb.2016017

[4]

Guido De Philippis, Antonio De Rosa, Jonas Hirsch. The area blow up set for bounded mean curvature submanifolds with respect to elliptic surface energy functionals. Discrete & Continuous Dynamical Systems - A, 2019, 39 (12) : 7031-7056. doi: 10.3934/dcds.2019243

[5]

Zhihua Zhang, Naoki Saito. PHLST with adaptive tiling and its application to antarctic remote sensing image approximation. Inverse Problems & Imaging, 2014, 8 (1) : 321-337. doi: 10.3934/ipi.2014.8.321

[6]

Hong Seng Sim, Wah June Leong, Chuei Yee Chen, Siti Nur Iqmal Ibrahim. Multi-step spectral gradient methods with modified weak secant relation for large scale unconstrained optimization. Numerical Algebra, Control & Optimization, 2018, 8 (3) : 377-387. doi: 10.3934/naco.2018024

[7]

Vladimir Georgiev, Sandra Lucente. Focusing nlkg equation with singular potential. Communications on Pure & Applied Analysis, 2018, 17 (4) : 1387-1406. doi: 10.3934/cpaa.2018068

[8]

Daoyin He, Ingo Witt, Huicheng Yin. On the strauss index of semilinear tricomi equation. Communications on Pure & Applied Analysis, 2020, 19 (10) : 4817-4838. doi: 10.3934/cpaa.2020213

[9]

Jia Cai, Guanglong Xu, Zhensheng Hu. Sketch-based image retrieval via CAT loss with elastic net regularization. Mathematical Foundations of Computing, 2020, 3 (4) : 219-227. doi: 10.3934/mfc.2020013

[10]

Diana Keller. Optimal control of a linear stochastic Schrödinger equation. Conference Publications, 2013, 2013 (special) : 437-446. doi: 10.3934/proc.2013.2013.437

[11]

Simone Cacace, Maurizio Falcone. A dynamic domain decomposition for the eikonal-diffusion equation. Discrete & Continuous Dynamical Systems - S, 2016, 9 (1) : 109-123. doi: 10.3934/dcdss.2016.9.109

[12]

Naeem M. H. Alkoumi, Pedro J. Torres. Estimates on the number of limit cycles of a generalized Abel equation. Discrete & Continuous Dynamical Systems - A, 2011, 31 (1) : 25-34. doi: 10.3934/dcds.2011.31.25

[13]

Wentao Huang, Jianlin Xiang. Soliton solutions for a quasilinear Schrödinger equation with critical exponent. Communications on Pure & Applied Analysis, 2016, 15 (4) : 1309-1333. doi: 10.3934/cpaa.2016.15.1309

[14]

Kin Ming Hui, Soojung Kim. Asymptotic large time behavior of singular solutions of the fast diffusion equation. Discrete & Continuous Dynamical Systems - A, 2017, 37 (11) : 5943-5977. doi: 10.3934/dcds.2017258

[15]

Nhu N. Nguyen, George Yin. Stochastic partial differential equation models for spatially dependent predator-prey equations. Discrete & Continuous Dynamical Systems - B, 2020, 25 (1) : 117-139. doi: 10.3934/dcdsb.2019175

[16]

Ademir Fernando Pazoto, Lionel Rosier. Uniform stabilization in weighted Sobolev spaces for the KdV equation posed on the half-line. Discrete & Continuous Dynamical Systems - B, 2010, 14 (4) : 1511-1535. doi: 10.3934/dcdsb.2010.14.1511

[17]

Zhiming Guo, Zhi-Chun Yang, Xingfu Zou. Existence and uniqueness of positive solution to a non-local differential equation with homogeneous Dirichlet boundary condition---A non-monotone case. Communications on Pure & Applied Analysis, 2012, 11 (5) : 1825-1838. doi: 10.3934/cpaa.2012.11.1825

[18]

Ka Luen Cheung, Man Chun Leung. Asymptotic behavior of positive solutions of the equation $ \Delta u + K u^{\frac{n+2}{n-2}} = 0$ in $IR^n$ and positive scalar curvature. Conference Publications, 2001, 2001 (Special) : 109-120. doi: 10.3934/proc.2001.2001.109

[Back to Top]