February  2021, 15(1): 1-25. doi: 10.3934/ipi.2020048

Reproducible kernel Hilbert space based global and local image segmentation

1. 

Centre for Mathematical Imaging Techniques and Department of Mathematical Sciences, University of Liverpool, United Kingdom

2. 

Department of Mathematics, Applied Mathematics and Statistics, Case Western Reserve, University, Cleveland, OH 44106, USA

3. 

Liverpool Vascular & Endovascular Service, Royal Liverpool and Broadgreen University, Hospitals NHS Trust, Liverpool, L7 8XP, United Kingdom

Received  December 2019 Revised  May 2020 Published  February 2021 Early access  August 2020

Image segmentation is the task of partitioning an image into individual objects, and has many important applications in a wide range of fields. The majority of segmentation methods rely on image intensity gradient to define edges between objects. However, intensity gradient fails to identify edges when the contrast between two objects is low. In this paper we aim to introduce methods to make such weak edges more prominent in order to improve segmentation results of objects of low contrast. This is done for two kinds of segmentation models: global and local. We use a combination of a reproducing kernel Hilbert space and approximated Heaviside functions to decompose an image and then show how this decomposition can be applied to a segmentation model. We show some results and robustness to noise, as well as demonstrating that we can combine the reconstruction and segmentation model together, allowing us to obtain both the decomposition and segmentation simultaneously.

Citation: Liam Burrows, Weihong Guo, Ke Chen, Francesco Torella. Reproducible kernel Hilbert space based global and local image segmentation. Inverse Problems and Imaging, 2021, 15 (1) : 1-25. doi: 10.3934/ipi.2020048
References:
[1]

L. Ambrosio and V. Tortorelli, On the approximation of free discontinuity problems, Bollettino Dell'Unione Matematica Italiana. B, 6 (1992), 105-123. 

[2]

L. Ambrosio and V. M. Tortorelli, Approximation of functional depending on jumps by elliptic functional via t-convergence, Communications on Pure and Applied Mathematics, 43 (1990), 999-1036.  doi: 10.1002/cpa.3160430805.

[3]

B. Appleton and H. Talbot, Globally minimal surfaces by continuous maximal flows, IEEE Transactions on Pattern Analysis and Machine Intelligence, 28 (2006), 106-118.  doi: 10.1109/TPAMI.2006.12.

[4]

N. Badshah and K. Chen, Image selective segmentation under geometrical constraints using an active contour approach, Communications in Computational Physics, 7 (2010), 759-778.  doi: 10.4208/cicp.2009.09.026.

[5]

E. BaeJ. Yuan and X.-C. Tai, Global minimization for continuous multiphase partitioning problems using a dual approach, International Journal of Computer Vision, 92 (2011), 112-129.  doi: 10.1007/s11263-010-0406-y.

[6]

Y. Boykov and V. Kolmogorov, An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision, Energy Minimization Methods in Computer Vision and Pattern Recognition, 2134 (2001), 359-374.  doi: 10.1007/3-540-44745-8_24.

[7]

Y. BoykovO. Veksler and R. Zabih, Fast approximate energy minimization via graph cuts, Proceedings of the Seventh IEEE International Conference on Computer Vision, 1 (1999), 377-384.  doi: 10.1109/ICCV.1999.791245.

[8]

X. BressonS. EsedoḡluP. VandergheynstJ.-P. Thiran and S. Osher, Fast global minimization of the active contour/snake model, Journal of Mathematical Imaging and Vision, 28 (2007), 151-167.  doi: 10.1007/s10851-007-0002-0.

[9]

X. CaiR. Chan and T. Zeng, A two-stage image segmentation method using a convex variant of the Mumford–Shah model and thresholding, SIAM Journal on Imaging Sciences, 6 (2013), 368-390.  doi: 10.1137/120867068.

[10]

V. CasellesR. Kimmel and G. Sapiro, Geodesic active contours, International Journal of Computer Vision, 22 (1997), 61-79.  doi: 10.1109/ICCV.1995.466871.

[11]

A. Chambolle, An algorithm for total variation minimization and applications, Journal of Mathematical Imaging and Vision, 20 (2004), 89-97. 

[12]

T. F. ChanS. Esedoglu and M. Nikolova, Algorithms for finding global minimizers of image segmentation and denoising models, SIAM Journal on Applied Mathematics, 66 (2006), 1632-1648.  doi: 10.1137/040615286.

[13]

T. F. Chan and L. A. Vese, Active contours without edges, IEEE Transactions on Image Processing, 10 (2001), 266-277.  doi: 10.1109/83.902291.

[14]

L.-J. DengW. Guo and T.-Z. Huang, Single-image super-resolution via an iterative reproducing kernel hilbert space method, IEEE Transactions on Circuits and Systems for Video Technology, 26 (2016), 2001-2014.  doi: 10.1109/TCSVT.2015.2475895.

[15]

C. GoutC. Le Guyader and L. Vese, Segmentation under geometrical conditions using geodesic active contours and interpolation using level set methods, Numerical Algorithms, 39 (2005), 155-173.  doi: 10.1007/s11075-004-3627-8.

[16]

J.-B. Hiriart-Urruty and C. Lemar{é}chal, Convex Analysis and Minimization Algorithms I: Fundamentals, volume 305., Springer Science & Business Media, 2013.

[17]

P. Jaccard, The distribution of flora in the apline zone.1, New Phytologist, 11 (1912), 37-50. 

[18]

S. H. KangB. Shafei and G. Steidl, Supervised and transductive multi-class segmentation using p-Laplacians and RKHS methods, Journal of Visual Communication and Image Representation, 25 (2014), 1136-1148.  doi: 10.1016/j.jvcir.2014.03.010.

[19]

M. KassA. Witkin and D. Terzopoulos, Snakes: Active contour models, International Journal of Computer Vision, 1 (1988), 321-331.  doi: 10.1007/BF00133570.

[20]

C. LiC.-Y. KaoJ. C. Gore and Z. Ding, Minimization of region-scalable fitting energy for image segmentation, IEEE Transactions on Image Processing, 17 (2008), 1940-1949.  doi: 10.1109/TIP.2008.2002304.

[21]

C. LiuM. Ng and T. Zeng, Weighted variational model for selective image segmentation with application to medical images, Pattern Recognition, 76 (2018), 367-379.  doi: 10.1016/j.patcog.2017.11.019.

[22]

Z. Li and F. Malgouyres, Regularized non-local total variation and application in image restoration, Journal of Mathematical Imaging and Vision, 59 (2017), 296-317.  doi: 10.1007/s10851-017-0732-6.

[23]

D. Mumford and J. Shah, Optimal approximations by piecewise smooth functions and associated variational problems, Communications on Pure and Applied Mathematics, 42 (1989), 577-685.  doi: 10.1002/cpa.3160420503.

[24]

S. Osher and J. A. Sethian, Fronts propagating with curvature-dependent speed: Algorithms based on hamilton-jacobi formulations, Journal of Computational Physics, 79 (1988), 12-49.  doi: 10.1016/0021-9991(88)90002-2.

[25]

R. B. Potts, Some generalized order-disorder transformations, Mathematical Proceedings of the Cambridge Philosophical Society, 48 (1952), 106-109.  doi: 10.1017/S0305004100027419.

[26]

M. RobertsK. Chen and K. L. Irion, A convex geodesic selective model for image segmentation, Journal of Mathematical Imaging and Vision, 61 (2019), 482-503.  doi: 10.1007/s10851-018-0857-2.

[27]

M. Roberts and J. Spencer, Chan–vese reformulation for selective image segmentation, Journal of Mathematical Imaging and Vision, 61 (2019), 1173-1196.  doi: 10.1007/s10851-019-00893-0.

[28]

J. Spencer and K. Chen, A convex and selective variational model for image segmentation, Communications in Mathematical Sciences, 13 (2015), 1453-1472.  doi: 10.4310/CMS.2015.v13.n6.a5.

[29]

X.-F. WangD.-S. Huang and H. Xu, An efficient local chan–vese model for image segmentation, Pattern Recognition, 43 (2010), 603-618.  doi: 10.1016/j.patcog.2009.08.002.

[30]

Y. Xu and W. Yin, A block coordinate descent method for regularized multiconvex optimization with applications to nonnegative tensor factorization and completion, SIAM Journal on Imaging Sciences, 6 (2013), 1758-1789.  doi: 10.1137/120887795.

[31]

J. Yuan, E. Bae, X.-C. Tai and Y. Boykov, A continuous max-flow approach to potts model, In European Conference on Computer Vision, Springer, 2010,379–392.

[32]

J. YuanE. BaeX.-C. Tai and Y. Boykov, A spatially continuous max-flow and min-cut framework for binary labeling problems, Numerische Mathematik, 126 (2014), 559-587.  doi: 10.1007/s00211-013-0569-x.

[33]

H. ZhangY. Chen and J. Shi, Nonparametric image segmentation using Renyi's statistical dependence measure, Journal of Mathematical Imaging and Vision, 44 (2012), 330-340.  doi: 10.1007/s10851-012-0329-z.

[34]

H. Zhao, A fast sweeping method for Eikonal equations, Mathematics of Computation, 74 (2005), 603-627.  doi: 10.1090/S0025-5718-04-01678-3.

show all references

References:
[1]

L. Ambrosio and V. Tortorelli, On the approximation of free discontinuity problems, Bollettino Dell'Unione Matematica Italiana. B, 6 (1992), 105-123. 

[2]

L. Ambrosio and V. M. Tortorelli, Approximation of functional depending on jumps by elliptic functional via t-convergence, Communications on Pure and Applied Mathematics, 43 (1990), 999-1036.  doi: 10.1002/cpa.3160430805.

[3]

B. Appleton and H. Talbot, Globally minimal surfaces by continuous maximal flows, IEEE Transactions on Pattern Analysis and Machine Intelligence, 28 (2006), 106-118.  doi: 10.1109/TPAMI.2006.12.

[4]

N. Badshah and K. Chen, Image selective segmentation under geometrical constraints using an active contour approach, Communications in Computational Physics, 7 (2010), 759-778.  doi: 10.4208/cicp.2009.09.026.

[5]

E. BaeJ. Yuan and X.-C. Tai, Global minimization for continuous multiphase partitioning problems using a dual approach, International Journal of Computer Vision, 92 (2011), 112-129.  doi: 10.1007/s11263-010-0406-y.

[6]

Y. Boykov and V. Kolmogorov, An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision, Energy Minimization Methods in Computer Vision and Pattern Recognition, 2134 (2001), 359-374.  doi: 10.1007/3-540-44745-8_24.

[7]

Y. BoykovO. Veksler and R. Zabih, Fast approximate energy minimization via graph cuts, Proceedings of the Seventh IEEE International Conference on Computer Vision, 1 (1999), 377-384.  doi: 10.1109/ICCV.1999.791245.

[8]

X. BressonS. EsedoḡluP. VandergheynstJ.-P. Thiran and S. Osher, Fast global minimization of the active contour/snake model, Journal of Mathematical Imaging and Vision, 28 (2007), 151-167.  doi: 10.1007/s10851-007-0002-0.

[9]

X. CaiR. Chan and T. Zeng, A two-stage image segmentation method using a convex variant of the Mumford–Shah model and thresholding, SIAM Journal on Imaging Sciences, 6 (2013), 368-390.  doi: 10.1137/120867068.

[10]

V. CasellesR. Kimmel and G. Sapiro, Geodesic active contours, International Journal of Computer Vision, 22 (1997), 61-79.  doi: 10.1109/ICCV.1995.466871.

[11]

A. Chambolle, An algorithm for total variation minimization and applications, Journal of Mathematical Imaging and Vision, 20 (2004), 89-97. 

[12]

T. F. ChanS. Esedoglu and M. Nikolova, Algorithms for finding global minimizers of image segmentation and denoising models, SIAM Journal on Applied Mathematics, 66 (2006), 1632-1648.  doi: 10.1137/040615286.

[13]

T. F. Chan and L. A. Vese, Active contours without edges, IEEE Transactions on Image Processing, 10 (2001), 266-277.  doi: 10.1109/83.902291.

[14]

L.-J. DengW. Guo and T.-Z. Huang, Single-image super-resolution via an iterative reproducing kernel hilbert space method, IEEE Transactions on Circuits and Systems for Video Technology, 26 (2016), 2001-2014.  doi: 10.1109/TCSVT.2015.2475895.

[15]

C. GoutC. Le Guyader and L. Vese, Segmentation under geometrical conditions using geodesic active contours and interpolation using level set methods, Numerical Algorithms, 39 (2005), 155-173.  doi: 10.1007/s11075-004-3627-8.

[16]

J.-B. Hiriart-Urruty and C. Lemar{é}chal, Convex Analysis and Minimization Algorithms I: Fundamentals, volume 305., Springer Science & Business Media, 2013.

[17]

P. Jaccard, The distribution of flora in the apline zone.1, New Phytologist, 11 (1912), 37-50. 

[18]

S. H. KangB. Shafei and G. Steidl, Supervised and transductive multi-class segmentation using p-Laplacians and RKHS methods, Journal of Visual Communication and Image Representation, 25 (2014), 1136-1148.  doi: 10.1016/j.jvcir.2014.03.010.

[19]

M. KassA. Witkin and D. Terzopoulos, Snakes: Active contour models, International Journal of Computer Vision, 1 (1988), 321-331.  doi: 10.1007/BF00133570.

[20]

C. LiC.-Y. KaoJ. C. Gore and Z. Ding, Minimization of region-scalable fitting energy for image segmentation, IEEE Transactions on Image Processing, 17 (2008), 1940-1949.  doi: 10.1109/TIP.2008.2002304.

[21]

C. LiuM. Ng and T. Zeng, Weighted variational model for selective image segmentation with application to medical images, Pattern Recognition, 76 (2018), 367-379.  doi: 10.1016/j.patcog.2017.11.019.

[22]

Z. Li and F. Malgouyres, Regularized non-local total variation and application in image restoration, Journal of Mathematical Imaging and Vision, 59 (2017), 296-317.  doi: 10.1007/s10851-017-0732-6.

[23]

D. Mumford and J. Shah, Optimal approximations by piecewise smooth functions and associated variational problems, Communications on Pure and Applied Mathematics, 42 (1989), 577-685.  doi: 10.1002/cpa.3160420503.

[24]

S. Osher and J. A. Sethian, Fronts propagating with curvature-dependent speed: Algorithms based on hamilton-jacobi formulations, Journal of Computational Physics, 79 (1988), 12-49.  doi: 10.1016/0021-9991(88)90002-2.

[25]

R. B. Potts, Some generalized order-disorder transformations, Mathematical Proceedings of the Cambridge Philosophical Society, 48 (1952), 106-109.  doi: 10.1017/S0305004100027419.

[26]

M. RobertsK. Chen and K. L. Irion, A convex geodesic selective model for image segmentation, Journal of Mathematical Imaging and Vision, 61 (2019), 482-503.  doi: 10.1007/s10851-018-0857-2.

[27]

M. Roberts and J. Spencer, Chan–vese reformulation for selective image segmentation, Journal of Mathematical Imaging and Vision, 61 (2019), 1173-1196.  doi: 10.1007/s10851-019-00893-0.

[28]

J. Spencer and K. Chen, A convex and selective variational model for image segmentation, Communications in Mathematical Sciences, 13 (2015), 1453-1472.  doi: 10.4310/CMS.2015.v13.n6.a5.

[29]

X.-F. WangD.-S. Huang and H. Xu, An efficient local chan–vese model for image segmentation, Pattern Recognition, 43 (2010), 603-618.  doi: 10.1016/j.patcog.2009.08.002.

[30]

Y. Xu and W. Yin, A block coordinate descent method for regularized multiconvex optimization with applications to nonnegative tensor factorization and completion, SIAM Journal on Imaging Sciences, 6 (2013), 1758-1789.  doi: 10.1137/120887795.

[31]

J. Yuan, E. Bae, X.-C. Tai and Y. Boykov, A continuous max-flow approach to potts model, In European Conference on Computer Vision, Springer, 2010,379–392.

[32]

J. YuanE. BaeX.-C. Tai and Y. Boykov, A spatially continuous max-flow and min-cut framework for binary labeling problems, Numerische Mathematik, 126 (2014), 559-587.  doi: 10.1007/s00211-013-0569-x.

[33]

H. ZhangY. Chen and J. Shi, Nonparametric image segmentation using Renyi's statistical dependence measure, Journal of Mathematical Imaging and Vision, 44 (2012), 330-340.  doi: 10.1007/s10851-012-0329-z.

[34]

H. Zhao, A fast sweeping method for Eikonal equations, Mathematics of Computation, 74 (2005), 603-627.  doi: 10.1090/S0025-5718-04-01678-3.

Figure 1.  Comparison of Geodesic and Euclidean distance constraints
Figure 2.  Left panel: 3D surface images of $ \psi $ of nine parameter pairs $ (\theta, c) $; right panel: the corresponding 2D images. In each panel, from left to right and then from top to bottom: $ ( \frac{4 \pi}{5} , \frac{51}{1024} ), ( \frac{4\pi}{5} , \frac{25}{64} ), ( \frac{4\pi}{5} , \frac{175}{256} ), ( \frac{6\pi}{5}, \frac{135}{1024} ), ( \frac{6\pi}{5}, \frac{1}{2} ), ( \frac{6\pi}{5}, \frac{25}{32} ), ( \frac{8\pi}{5}, \frac{5}{64} ) $, $ ( \frac{8\pi}{5}, \frac{75}{256} ), ( \frac{8\pi}{5}, \frac{75}{128} ) $
Figure 3.  The decomposition of the input image (top left), into the smooth parts (top middle) and edge parts (top right), obtained from RKHS model (8), and edge indicator function $ g(\Psi \beta) $ (bottom left). In comparison, we display the gradient of the image, $ |\nabla z| $ and the edge indicator function $ g(|\nabla z|) $ in the bottom middle and bottom right respectively
Figure 4.  Segmentation of white matter from MRI brain image. We show both the contour overlaid on the original image, and the associated binary segmentation result for each example. First column: segmentation using edge weighted Chan-Vese using $ |\nabla z| $ model (24). Second column: Edge weighted Chan-Vese model (24) using Canny ($ g = g_{\mathcal{C}} $). Third column: Cai et al. [9]. Fourth column: Proposed two stage global model (8) and (25). Final column: The proposed combined global segmentation model (26)
Figure 5.  Row 1: 3% noise, Row 2: 5% noise, Row 3: 7% noise, Row 4: 9% noise. We show both the contour overlaid on the original image, and the associated binary segmentation result for each example. First two columns: Using model (24). Final two columns: Combined model, algorithm (1)
Figure 6.  Geodesic distances computed using gradient (middle) and $ \Psi \beta $ (right)
Figure 7.  Row one: Original clean image and the user input. Row two: Image and result with 20% noise. Row three: Image and result with 40% noise. Row four: Image and result with 60% noise
Figure 8.  Segmentation of blood vessel
Figure 9.  Segmentation of blood vessel
Figure 10.  Result of the combined selective model
[1]

Fan Jia, Xue-Cheng Tai, Jun Liu. Nonlocal regularized CNN for image segmentation. Inverse Problems and Imaging, 2020, 14 (5) : 891-911. doi: 10.3934/ipi.2020041

[2]

Ye Yuan, Yan Ren, Xiaodong Liu, Jing Wang. Approach to image segmentation based on interval neutrosophic set. Numerical Algebra, Control and Optimization, 2020, 10 (1) : 1-11. doi: 10.3934/naco.2019028

[3]

Dominique Zosso, Jing An, James Stevick, Nicholas Takaki, Morgan Weiss, Liane S. Slaughter, Huan H. Cao, Paul S. Weiss, Andrea L. Bertozzi. Image segmentation with dynamic artifacts detection and bias correction. Inverse Problems and Imaging, 2017, 11 (3) : 577-600. doi: 10.3934/ipi.2017027

[4]

Matthew S. Keegan, Berta Sandberg, Tony F. Chan. A multiphase logic framework for multichannel image segmentation. Inverse Problems and Imaging, 2012, 6 (1) : 95-110. doi: 10.3934/ipi.2012.6.95

[5]

Shi Yan, Jun Liu, Haiyang Huang, Xue-Cheng Tai. A dual EM algorithm for TV regularized Gaussian mixture model in image segmentation. Inverse Problems and Imaging, 2019, 13 (3) : 653-677. doi: 10.3934/ipi.2019030

[6]

Jianping Zhang, Ke Chen, Bo Yu, Derek A. Gould. A local information based variational model for selective image segmentation. Inverse Problems and Imaging, 2014, 8 (1) : 293-320. doi: 10.3934/ipi.2014.8.293

[7]

Lu Tan, Ling Li, Senjian An, Zhenkuan Pan. Nonlinear diffusion based image segmentation using two fast algorithms. Mathematical Foundations of Computing, 2019, 2 (2) : 149-168. doi: 10.3934/mfc.2019011

[8]

Ruiliang Zhang, Xavier Bresson, Tony F. Chan, Xue-Cheng Tai. Four color theorem and convex relaxation for image segmentation with any number of regions. Inverse Problems and Imaging, 2013, 7 (3) : 1099-1113. doi: 10.3934/ipi.2013.7.1099

[9]

Balázs Kósa, Karol Mikula, Markjoe Olunna Uba, Antonia Weberling, Neophytos Christodoulou, Magdalena Zernicka-Goetz. 3D image segmentation supported by a point cloud. Discrete and Continuous Dynamical Systems - S, 2021, 14 (3) : 971-985. doi: 10.3934/dcdss.2020351

[10]

Jie Huang, Xiaoping Yang, Yunmei Chen. A fast algorithm for global minimization of maximum likelihood based on ultrasound image segmentation. Inverse Problems and Imaging, 2011, 5 (3) : 645-657. doi: 10.3934/ipi.2011.5.645

[11]

Maika Goto, Kazunori Kuwana, Yasuhide Uegata, Shigetoshi Yazaki. A method how to determine parameters arising in a smoldering evolution equation by image segmentation for experiment's movies. Discrete and Continuous Dynamical Systems - S, 2021, 14 (3) : 881-891. doi: 10.3934/dcdss.2020233

[12]

Baoli Shi, Zhi-Feng Pang, Jing Xu. Image segmentation based on the hybrid total variation model and the K-means clustering strategy. Inverse Problems and Imaging, 2016, 10 (3) : 807-828. doi: 10.3934/ipi.2016022

[13]

Yupeng Li, Wuchen Li, Guo Cao. Image segmentation via $ L_1 $ Monge-Kantorovich problem. Inverse Problems and Imaging, 2019, 13 (4) : 805-826. doi: 10.3934/ipi.2019037

[14]

Qianting Ma, Tieyong Zeng, Dexing Kong, Jianwei Zhang. Weighted area constraints-based breast lesion segmentation in ultrasound image analysis. Inverse Problems and Imaging, 2022, 16 (2) : 451-466. doi: 10.3934/ipi.2021057

[15]

Kateřina Škardová, Tomáš Oberhuber, Jaroslav Tintěra, Radomír Chabiniok. Signed-distance function based non-rigid registration of image series with varying image intensity. Discrete and Continuous Dynamical Systems - S, 2021, 14 (3) : 1145-1160. doi: 10.3934/dcdss.2020386

[16]

Ibrar Hussain, Haider Ali, Muhammad Shahkar Khan, Sijie Niu, Lavdie Rada. Robust region-based active contour models via local statistical similarity and local similarity factor for intensity inhomogeneity and high noise image segmentation. Inverse Problems and Imaging, , () : -. doi: 10.3934/ipi.2022014

[17]

Yuantian Xia, Juxiang Zhou, Tianwei Xu, Wei Gao. An improved deep convolutional neural network model with kernel loss function in image classification. Mathematical Foundations of Computing, 2020, 3 (1) : 51-64. doi: 10.3934/mfc.2020005

[18]

Xiaoli Wang, Meihua Yang, Peter E. Kloeden. Sigmoidal approximations of a delay neural lattice model with Heaviside functions. Communications on Pure and Applied Analysis, 2020, 19 (4) : 2385-2402. doi: 10.3934/cpaa.2020104

[19]

Jun Fan, Fusheng Lv, Lei Shi. An RKHS approach to estimate individualized treatment rules based on functional predictors. Mathematical Foundations of Computing, 2019, 2 (2) : 169-181. doi: 10.3934/mfc.2019012

[20]

He Zhang, John Harlim, Xiantao Li. Estimating linear response statistics using orthogonal polynomials: An RKHS formulation. Foundations of Data Science, 2020, 2 (4) : 443-485. doi: 10.3934/fods.2020021

2020 Impact Factor: 1.639

Metrics

  • PDF downloads (456)
  • HTML views (307)
  • Cited by (0)

[Back to Top]