American Institute of Mathematical Sciences

Advanced
June  2017, 11(3): 577-600. doi: 10.3934/ipi.2017027

Image segmentation with dynamic artifacts detection and bias correction

Dominique Zosso 1,1, , Jing An 1, , James Stevick 1, , Nicholas Takaki 1, , Morgan Weiss 1, , Liane S. Slaughter 2,3,4, , Huan H. Cao 2,3,4, , Paul S. Weiss 2,3,4, and  Andrea L. Bertozzi 5,

1. 

University of California, Los Angeles, Department of Mathematics, 520 Portola Plaza, Box 951555, Los Angeles, CA 90095-1555, USA
2. 

University of California, Los Angeles, California NanoSystems Institute (CNSI), 570 Westwood Plaza, Building 114, Los Angeles, CA 90095, USA
3. 

University of California, Los Angeles, Department of Chemistry and Biochemistry, 607 Charles E. Young Drive, Los Angeles, CA 90095, USA
4. 

University of California, Los Angeles, Department of Materials Science and Engineering, 410 Westwood Plaza, Los Angeles, CA 90095, USA
5. 

University of California, Los Angeles, Department of Mathematics, 520 Portola Plaza, Box 951555, Los Angeles, CA 90095-1555, USA

1 To whom correspondence should be addressed. Current affiliation and address: Montana State University, Department of Mathematical Sciences, Wilson Hall 2-214, P.O. Box 172400, Bozeman, MT 59717-2400, USA

Received  January 2015 Revised  February 2017 Published  April 2017

Fund Project: This work is supported by the Swiss National Science Foundation under grant P300P2-147778, the California Research Training Program in Computational and Applied Mathematics under grant NSF DMS-1045536, the W. M. Keck Foundation, ONR N00014-16-1-2119, and the Merkin Family Foundation.

Region-based image segmentation is well-addressed by the Chan-Vese (CV) model. However, this approach fails when images are affected by artifacts (outliers) and illumination bias that outweigh the actual image contrast. Here, we introduce a model for segmenting such images. In a single energy functional, we introduce 1) a dynamic artifact class preventing intensity outliers from skewing the segmentation, and 2), in Retinex-fashion, we decompose the image into a piecewise-constant structural part and a smooth bias part. The CV-segmentation terms then only act on the structure, and only in regions not identified as artifacts. The segmentation is parameterized using a phase-field, and efficiently minimized using threshold dynamics.

We demonstrate the proposed model on a series of sample images from diverse modalities exhibiting artifacts and/or bias. Our algorithm typically converges within 10-50 iterations and takes fractions of a second on standard equipment to produce meaningful results. We expect our method to be useful for damaged images, and anticipate use in applications where artifacts and bias are actual features of interest, such as lesion detection and bias field correction in medical imaging, e.g., in magnetic resonance imaging (MRI).

Keywords: Image segmentation, artifacts, bias, Retinex, dynamic labeling, ChanVese, levelset method, threshold dynamics, MBO.
Mathematics Subject Classification: Primary: 68U10, 68T45; Secondary: 49N45, 65K10, 35A15, 35Q93.
Citation: Dominique Zosso, Jing An, James Stevick, Nicholas Takaki, Morgan Weiss, Liane S. Slaughter, Huan H. Cao, Paul S. Weiss, Andrea L. Bertozzi. Image segmentation with dynamic artifacts detection and bias correction. Inverse Problems and Imaging, 2017, 11 (3) : 577-600. doi: 10.3934/ipi.2017027
References:
[1]

A. Ayvaci, M. Raptis and S. Soatto, Sparse occlusion detection with optical flow, International Journal of Computer Vision, 97 (2012), 322-338.  doi: 10.1007/s11263-011-0490-7.

[2]

X. Bresson, S. Esedoglu, P. Vandergheynst, J.-P. Thiran and S. Osher, Fast global minimization of the active contour/snake model, Journal of Mathematical Imaging and Vision, 28 (2007), 151-167.  doi: 10.1007/s10851-007-0002-0.

[3]

T. Brox and D. Cremers, On local region models and a statistical interpretation of the piecewise smooth mumford-shah functional, International Journal of Computer Vision, 84 (2008), 184-193.  doi: 10.1007/s11263-008-0153-5.

[4]

V. Caselles, R. Kimmel and G. Sapiro, Geodesic active contours, Int. J. Comput. Vis., 22 (1995), 61-79.  doi: 10.1109/ICCV.1995.466871.

[5]

T. F. Chan and L. A. Vese, Active contours without edges, IEEE Transactions on Image Processing, 10 (2001), 266-277.  doi: 10.1109/83.902291.

[6]

T. Chan and W. Zhu, Level set based shape prior segmentation, , in CVPR 2005, IEEE, 2 (2005), 1164-1170.  doi: 10.1109/CVPR.2005.212.

[7]

T. F. Chan, S. Esedoglu and M. Nikolova, Algorithms for finding global minimizers of image segmentation and denoising models, SIAM Journal on Applied Mathematics, 66 (2006), 1632-1648.  doi: 10.1137/040615286.

[8]

T. F. Chan, B. Sandberg and L. A. Vese, Active contours without edges for vector-valued images, Journal of Visual Communication and Image Representation, 11 (2000), 130-141.  doi: 10.1006/jvci.1999.0442.

[9]

S. A. Claridge, W.-S. Liao, J. C. Thomas, Y. Zhao, H. H. Cao, S. Cheunkar, A. C. Serino, A. M. Andrews and P. S. Weiss, From the bottom up: Dimensional control and characterization in molecular monolayers, Chem. Soc. Rev., 42 (2013), 2725-2745.  doi: 10.1039/C2CS35365B.

[10]

D. Cremers, S. J. Osher and S. Soatto, Kernel density estimation and intrinsic alignment for shape priors in level set segmentation, International Journal of Computer Vision, 69 (2006), 335-351.  doi: 10.1007/s11263-006-7533-5.

[11]

D. Cremers, N. Sochen and C. Schnörr, A multiphase dynamic labeling model for variational recognition-driven image segmentation, International Journal of Computer Vision, 66 (2006), 67-81.  doi: 10.1007/s11263-005-3676-z.

[12]

S. Esedoglu and F. Otto, Threshold dynamics for networks with arbitrary surface tensions, Communications on Pure and Applied Mathematics, 68 (2015), 808-864.  doi: 10.1002/cpa.21527.

[13]

S. Esedoglu and Y. H. R. Tsai, Threshold dynamics for the piecewise constant Mumford-Shah functional, Journal of Computational Physics, 211 (2006), 367-384.  doi: 10.1016/j.jcp.2005.05.027.

[14]

V. Estellers and S. Soatto, Detecting occlusions as an inverse problem, Journal of Mathematical Imaging and Vision, 54 (2016), 181-198.  doi: 10.1007/s10851-015-0596-6.

[15]

V. Estellers, D. Zosso, R. Lai, S. Osher, J.-P. Thiran and X. Bresson, Efficient algorithm for level set method preserving distance function, IEEE Transactions on Image Processing, 21 (2012), 4722-4734.  doi: 10.1109/TIP.2012.2202674.

[16]

P. Filzmoser, R. G. Garrett and C. Reimann, Multivariate outlier detection in exploration geochemistry, Computers & Geosciences, 31 (2005), 579-587.  doi: 10.1016/j.cageo.2004.11.013.

[17]

B. K. Horn, Determining lightness from an image, Computer Graphics and Image Processing, 3 (1974), 277-299.  doi: 10.1016/0146-664X(74)90022-7.

[18]

M. Jung, M. Kang and M. Kang, Variational image segmentation models involving non-smooth data-fidelity terms, Journal of Scientific Computing, 59 (2013), 277-308.  doi: 10.1007/s10915-013-9766-0.

[19]

M. Kass, A. Witkin and D. Terzopoulos, Snakes: Active contour models, International Journal of Computer Vision, 1 (1988), 321-331.  doi: 10.1007/BF00133570.

[20]

R. Kimmel, M. Elad, D. Shaked, R. Keshet and I. Sobel, A variational framework for Retinex, International Journal of Computer Vision, 52 (2003), 7-23. 

[21]

E. H. Land, The retinex, American Scientist, 52 (1964), 247-264. 

[22]

E. H. Land, The Retinex theory of color vision, Scientific American, 237 (1977), 108-128. 

[23]

E. H. Land and J. J. McCann, Lightness and Retinex theory, Journal of the Optical Society of America, 61 (1971), 1-11. 

[24]

C. Li, C.-Y. Kao, J. C. Gore and Z. Ding, Minimization of region-scalable fitting energy for image segmentation, IEEE Transactions on Image Processing, 17 (2008), 1940-1949.  doi: 10.1109/TIP.2008.2002304.

[25]

C. Li, C. Xu, C. Gui and M. D. Fox, Level set evolution without re-initialization: A new variational formulation, in IEEE CVPR, IEEE, 1 (2005), 430-436. 

[26]

C. Li, C. Xu, C. Gui and M. D. Fox, Distance regularized level set evolution and its application to image segmentation, IEEE Transactions on Image Processing, 19 (2010), 3243-3254.  doi: 10.1109/TIP.2010.2069690.

[27]

C. Li, C. Xu, K. M. Konwar and M. D. Fox, Fast distance preserving level set evolution for medical image segmentation, in 2006 9th International Conference on Control, Automation, Robotics and Vision, IEEE, 2006, 1–7. doi: 10.1109/ICARCV.2006.345357.

[28]

F. Li, S. Osher, J. Qin and M. Yan, A multiphase image segmentation based on fuzzy membership functions and L1-norm fidelity, J. Sci. Comput., 69 (2016), 82–106, URL http://arxiv.org/abs/1504.02206. doi: 10.1007/s10915-016-0183-z.

[29]

W. Ma and S. Osher, A TV Bregman iterative model of Retinex theory, Inverse Problems and Imaging (IPI), 6 (2012), 697-708.  doi: 10.3934/ipi.2012.6.697.

[30]

R. Madani, A. Bourquard and M. Unser, Image segmentation with background correction using a multiplicative smoothing-spline model, in 2012 9th IEEE International Symposium on Biomedical Imaging (ISBI), IEEE, 2012, 186–189. doi: 10.1109/ISBI.2012.6235515.

[31]

B. Merriman, J. K. Bence and S. Osher, Diffusion Generated Motion by Mean Curvature, Technical report, UCLA CAM Report 92-18, 1992.

[32]

B. Merriman, J. K. Bence and S. J. Osher, Motion of multiple junctions: A level set approach, Journal of Computational Physics, 112 (1994), 334-363.  doi: 10.1006/jcph.1994.1105.

[33]

L. Modica, The gradient theory of phase transitions and the minimal interface criterion, Archive for Rational Mechanics and Analysis, 98 (1987), 123-142.  doi: 10.1007/BF00251230.

[34]

J.-M. Morel, A.-B. Petro and C. Sbert, A PDE formalization of Retinex theory, IEEE Transactions on Image Processing, 19 (2010), 2825-2837.  doi: 10.1109/TIP.2010.2049239.

[35]

D. Mumford and J. Shah, Optimal approximations by piecewise smooth functions and associated variational problems, Communications on Pure and Applied Mathematics, 42 (1989), 577-685.  doi: 10.1002/cpa.3160420503.

[36]

J. Nocedal and S. J. Wright, Numerical Optimization, 2nd edition, Springer, Berlin, 2006.

[37]

S. Osher and J. A. Sethian, Fronts propagating with curvature-dependent speed: Algorithms based on Hamilton-Jacobi formulations, Journal of Computational Physics, 79 (1988), 12-49.  doi: 10.1016/0021-9991(88)90002-2.

[38]

M. Prastawa, E. Bullitt, S. Ho and G. Gerig, A brain tumor segmentation framework based on outlier detection., Medical Image Analysis, 8 (2004), 275-283.  doi: 10.1016/j.media.2004.06.007.

[39]

M. Rousson and N. Paragios, Shape Priors for Level Set Representations, in ECCV 2002 (eds. A. Heyden, G. Sparr, M. Nielsen and P. Johansen), vol. 2351 of Lecture Notes in Computer Science, Springer Berlin Heidelberg, Berlin, Heidelberg, 2002, 78–92. doi: 10.1007/3-540-47967-8_6.

[40]

S. Ruuth, Efficient algorithms for diffusion-generated motion by mean curvature, Journal of Computational Physics, 144 (1998), 603-625.  doi: 10.1006/jcph.1998.6025.

[41]

Y. van Gennip, N. Guillen, B. Osting and A. L. Bertozzi, Mean curvature, threshold dynamics, and phase field theory on finite graphs, Milan Journal of Mathematics, 82 (2014), 3-65.  doi: 10.1007/s00032-014-0216-8.

[42]

L. A. Vese and T. F. Chan, A multiphase level set framework for image segmentation using the Mumford and Shah model, International Journal of Computer Vision, 50 (2002), 271-293. 

[43]

L. Wang, C. Li, Q. Sun, D. Xia and C.-Y. Kao, Active contours driven by local and global intensity fitting energy with application to brain MR image segmentation, Computerized medical imaging and graphics, 33 (2009), 520-531.  doi: 10.1016/j.compmedimag.2009.04.010.

[44]

P. S. Weiss, Functional molecules and assemblies in controlled environments: Formation and measurements, Accounts of Chemical Research, 41 (2008), 1772-1781.  doi: 10.1021/ar8001443.

[45]

M. Yan, Restoration of images corrupted by impulse noise and mixed Gaussian impulse noise using blind inpainting, SIAM Journal on Imaging Sciences, 6 (2013), 1227-1245.  doi: 10.1137/12087178X.

[46]

Y. Yang, C. Li, C. -Y. Kao and S. Osher, Split bregman method for minimization of regionscalable fitting energy for image segmentation, in Advances in Visual Computing, vol. 6454 of Lecture Notes in Computer Science, Springer Berlin Heidelberg, 2010, 117–128.

[47]

D. Zosso, G. Tran and S. Osher, A unifying Retinex model based on non-local differential operators in IS & T/SPIE Electronic Imaging (ed. C. A. Bouman), 8657 (2013), 865702. doi: 10.1117/12.2008839.

[48]

D. Zosso, G. Tran and S. J. Osher, Non-local Retinex—a unifying framework and beyond, SIAM J. Imaging Sciences, 8 (2015), 787-826.  doi: 10.1137/140972664.

show all references

References:
[1]

A. Ayvaci, M. Raptis and S. Soatto, Sparse occlusion detection with optical flow, International Journal of Computer Vision, 97 (2012), 322-338.  doi: 10.1007/s11263-011-0490-7.

[2]

X. Bresson, S. Esedoglu, P. Vandergheynst, J.-P. Thiran and S. Osher, Fast global minimization of the active contour/snake model, Journal of Mathematical Imaging and Vision, 28 (2007), 151-167.  doi: 10.1007/s10851-007-0002-0.

[3]

T. Brox and D. Cremers, On local region models and a statistical interpretation of the piecewise smooth mumford-shah functional, International Journal of Computer Vision, 84 (2008), 184-193.  doi: 10.1007/s11263-008-0153-5.

[4]

V. Caselles, R. Kimmel and G. Sapiro, Geodesic active contours, Int. J. Comput. Vis., 22 (1995), 61-79.  doi: 10.1109/ICCV.1995.466871.

[5]

T. F. Chan and L. A. Vese, Active contours without edges, IEEE Transactions on Image Processing, 10 (2001), 266-277.  doi: 10.1109/83.902291.

[6]

T. Chan and W. Zhu, Level set based shape prior segmentation, , in CVPR 2005, IEEE, 2 (2005), 1164-1170.  doi: 10.1109/CVPR.2005.212.

[7]

T. F. Chan, S. Esedoglu and M. Nikolova, Algorithms for finding global minimizers of image segmentation and denoising models, SIAM Journal on Applied Mathematics, 66 (2006), 1632-1648.  doi: 10.1137/040615286.

[8]

T. F. Chan, B. Sandberg and L. A. Vese, Active contours without edges for vector-valued images, Journal of Visual Communication and Image Representation, 11 (2000), 130-141.  doi: 10.1006/jvci.1999.0442.

[9]

S. A. Claridge, W.-S. Liao, J. C. Thomas, Y. Zhao, H. H. Cao, S. Cheunkar, A. C. Serino, A. M. Andrews and P. S. Weiss, From the bottom up: Dimensional control and characterization in molecular monolayers, Chem. Soc. Rev., 42 (2013), 2725-2745.  doi: 10.1039/C2CS35365B.

[10]

D. Cremers, S. J. Osher and S. Soatto, Kernel density estimation and intrinsic alignment for shape priors in level set segmentation, International Journal of Computer Vision, 69 (2006), 335-351.  doi: 10.1007/s11263-006-7533-5.

[11]

D. Cremers, N. Sochen and C. Schnörr, A multiphase dynamic labeling model for variational recognition-driven image segmentation, International Journal of Computer Vision, 66 (2006), 67-81.  doi: 10.1007/s11263-005-3676-z.

[12]

S. Esedoglu and F. Otto, Threshold dynamics for networks with arbitrary surface tensions, Communications on Pure and Applied Mathematics, 68 (2015), 808-864.  doi: 10.1002/cpa.21527.

[13]

S. Esedoglu and Y. H. R. Tsai, Threshold dynamics for the piecewise constant Mumford-Shah functional, Journal of Computational Physics, 211 (2006), 367-384.  doi: 10.1016/j.jcp.2005.05.027.

[14]

V. Estellers and S. Soatto, Detecting occlusions as an inverse problem, Journal of Mathematical Imaging and Vision, 54 (2016), 181-198.  doi: 10.1007/s10851-015-0596-6.

[15]

V. Estellers, D. Zosso, R. Lai, S. Osher, J.-P. Thiran and X. Bresson, Efficient algorithm for level set method preserving distance function, IEEE Transactions on Image Processing, 21 (2012), 4722-4734.  doi: 10.1109/TIP.2012.2202674.

[16]

P. Filzmoser, R. G. Garrett and C. Reimann, Multivariate outlier detection in exploration geochemistry, Computers & Geosciences, 31 (2005), 579-587.  doi: 10.1016/j.cageo.2004.11.013.

[17]

B. K. Horn, Determining lightness from an image, Computer Graphics and Image Processing, 3 (1974), 277-299.  doi: 10.1016/0146-664X(74)90022-7.

[18]

M. Jung, M. Kang and M. Kang, Variational image segmentation models involving non-smooth data-fidelity terms, Journal of Scientific Computing, 59 (2013), 277-308.  doi: 10.1007/s10915-013-9766-0.

[19]

M. Kass, A. Witkin and D. Terzopoulos, Snakes: Active contour models, International Journal of Computer Vision, 1 (1988), 321-331.  doi: 10.1007/BF00133570.

[20]

R. Kimmel, M. Elad, D. Shaked, R. Keshet and I. Sobel, A variational framework for Retinex, International Journal of Computer Vision, 52 (2003), 7-23. 

[21]

E. H. Land, The retinex, American Scientist, 52 (1964), 247-264. 

[22]

E. H. Land, The Retinex theory of color vision, Scientific American, 237 (1977), 108-128. 

[23]

E. H. Land and J. J. McCann, Lightness and Retinex theory, Journal of the Optical Society of America, 61 (1971), 1-11. 

[24]

C. Li, C.-Y. Kao, J. C. Gore and Z. Ding, Minimization of region-scalable fitting energy for image segmentation, IEEE Transactions on Image Processing, 17 (2008), 1940-1949.  doi: 10.1109/TIP.2008.2002304.

[25]

C. Li, C. Xu, C. Gui and M. D. Fox, Level set evolution without re-initialization: A new variational formulation, in IEEE CVPR, IEEE, 1 (2005), 430-436. 

[26]

C. Li, C. Xu, C. Gui and M. D. Fox, Distance regularized level set evolution and its application to image segmentation, IEEE Transactions on Image Processing, 19 (2010), 3243-3254.  doi: 10.1109/TIP.2010.2069690.

[27]

C. Li, C. Xu, K. M. Konwar and M. D. Fox, Fast distance preserving level set evolution for medical image segmentation, in 2006 9th International Conference on Control, Automation, Robotics and Vision, IEEE, 2006, 1–7. doi: 10.1109/ICARCV.2006.345357.

[28]

F. Li, S. Osher, J. Qin and M. Yan, A multiphase image segmentation based on fuzzy membership functions and L1-norm fidelity, J. Sci. Comput., 69 (2016), 82–106, URL http://arxiv.org/abs/1504.02206. doi: 10.1007/s10915-016-0183-z.

[29]

W. Ma and S. Osher, A TV Bregman iterative model of Retinex theory, Inverse Problems and Imaging (IPI), 6 (2012), 697-708.  doi: 10.3934/ipi.2012.6.697.

[30]

R. Madani, A. Bourquard and M. Unser, Image segmentation with background correction using a multiplicative smoothing-spline model, in 2012 9th IEEE International Symposium on Biomedical Imaging (ISBI), IEEE, 2012, 186–189. doi: 10.1109/ISBI.2012.6235515.

[31]

B. Merriman, J. K. Bence and S. Osher, Diffusion Generated Motion by Mean Curvature, Technical report, UCLA CAM Report 92-18, 1992.

[32]

B. Merriman, J. K. Bence and S. J. Osher, Motion of multiple junctions: A level set approach, Journal of Computational Physics, 112 (1994), 334-363.  doi: 10.1006/jcph.1994.1105.

[33]

L. Modica, The gradient theory of phase transitions and the minimal interface criterion, Archive for Rational Mechanics and Analysis, 98 (1987), 123-142.  doi: 10.1007/BF00251230.

[34]

J.-M. Morel, A.-B. Petro and C. Sbert, A PDE formalization of Retinex theory, IEEE Transactions on Image Processing, 19 (2010), 2825-2837.  doi: 10.1109/TIP.2010.2049239.

[35]

D. Mumford and J. Shah, Optimal approximations by piecewise smooth functions and associated variational problems, Communications on Pure and Applied Mathematics, 42 (1989), 577-685.  doi: 10.1002/cpa.3160420503.

[36]

J. Nocedal and S. J. Wright, Numerical Optimization, 2nd edition, Springer, Berlin, 2006.

[37]

S. Osher and J. A. Sethian, Fronts propagating with curvature-dependent speed: Algorithms based on Hamilton-Jacobi formulations, Journal of Computational Physics, 79 (1988), 12-49.  doi: 10.1016/0021-9991(88)90002-2.

[38]

M. Prastawa, E. Bullitt, S. Ho and G. Gerig, A brain tumor segmentation framework based on outlier detection., Medical Image Analysis, 8 (2004), 275-283.  doi: 10.1016/j.media.2004.06.007.

[39]

M. Rousson and N. Paragios, Shape Priors for Level Set Representations, in ECCV 2002 (eds. A. Heyden, G. Sparr, M. Nielsen and P. Johansen), vol. 2351 of Lecture Notes in Computer Science, Springer Berlin Heidelberg, Berlin, Heidelberg, 2002, 78–92. doi: 10.1007/3-540-47967-8_6.

[40]

S. Ruuth, Efficient algorithms for diffusion-generated motion by mean curvature, Journal of Computational Physics, 144 (1998), 603-625.  doi: 10.1006/jcph.1998.6025.

[41]

Y. van Gennip, N. Guillen, B. Osting and A. L. Bertozzi, Mean curvature, threshold dynamics, and phase field theory on finite graphs, Milan Journal of Mathematics, 82 (2014), 3-65.  doi: 10.1007/s00032-014-0216-8.

[42]

L. A. Vese and T. F. Chan, A multiphase level set framework for image segmentation using the Mumford and Shah model, International Journal of Computer Vision, 50 (2002), 271-293. 

[43]

L. Wang, C. Li, Q. Sun, D. Xia and C.-Y. Kao, Active contours driven by local and global intensity fitting energy with application to brain MR image segmentation, Computerized medical imaging and graphics, 33 (2009), 520-531.  doi: 10.1016/j.compmedimag.2009.04.010.

[44]

P. S. Weiss, Functional molecules and assemblies in controlled environments: Formation and measurements, Accounts of Chemical Research, 41 (2008), 1772-1781.  doi: 10.1021/ar8001443.

[45]

M. Yan, Restoration of images corrupted by impulse noise and mixed Gaussian impulse noise using blind inpainting, SIAM Journal on Imaging Sciences, 6 (2013), 1227-1245.  doi: 10.1137/12087178X.

[46]

Y. Yang, C. Li, C. -Y. Kao and S. Osher, Split bregman method for minimization of regionscalable fitting energy for image segmentation, in Advances in Visual Computing, vol. 6454 of Lecture Notes in Computer Science, Springer Berlin Heidelberg, 2010, 117–128.

[47]

D. Zosso, G. Tran and S. Osher, A unifying Retinex model based on non-local differential operators in IS & T/SPIE Electronic Imaging (ed. C. A. Bouman), 8657 (2013), 865702. doi: 10.1117/12.2008839.

[48]

D. Zosso, G. Tran and S. J. Osher, Non-local Retinex—a unifying framework and beyond, SIAM J. Imaging Sciences, 8 (2015), 787-826.  doi: 10.1137/140972664.

Figure 1.  Example cases 1 & 2. Top: Coronal MRI slice. Input image and initial segmentation contour are shown in the top-right corner. The image is heavily affected by intensity bias, such that the classical CV model fails. The superior parts of the white-matter are undersegmented, while the inferior regions are markedly oversegmented. CV+X is not very helpful, here. In contrast, CV+B fixes the problem: the extracted structure is nearly flat (the brain is essentially two-phase piecewise constant), while CV+XB marks some non-brain pixels as outliers. Bottom: This synthethic image is a combination of piecewise constant regions affected by strong noise and oscillating bias. Again, CV fails, and artifacts detection not appropriate. Bias correction greatly improves the segmentation, but errors persist since from the simple initialization the algorithm converges to a wrong local minimum (upper part of left hand structure)
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 2.  Example cases 3 & 4. Top: The seemingly simple scene is not segmentable by the CV model alone, due to strong bias. CV+X wrongly classifies bright regions as artifacts. Bias correction (CV+B) results in accurate segmentation of the T-object. Bottom: The vessel structure is not accurately segmented by classical CV: superior parts are oversegmented due to brightening, inferior parts are undersegmented due to darkening. CV+X is inappropriate, while CV+B fixes the problem and leads to much improved vessel segmentation
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 3.  Example cases 5 & 6. Top: Compare to ex. 4 in figure 2.Bottom: Three-phase piecewise constant synthetic image. The goal is to separate the two black ellipses from the gray background, considering the white ring to be an occlusion artifact. CV, however, groups the white ring with the light background. CV+X successfully identifies the ring as artifact, and closes the black ellipses thanks to the interface regularization. The bias correction is misleading, since much of the white ring will be considered overly illuminated background (CV+B, CV+XB), the corners being captured as transient artifacts (CV+XB)
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 4.  Results by region-scalable-fitting (RSF) [24]. Left: Starting from generic initialization (as used in our method), RSF fails to capture the correct bias/segmentation result.Right: Starting from a tuned initialization (as provided in [24]), RSF produces the desired bias resistant region-based image segmentation result. Note that for optimally chosen number of iterations, $i$ (as provided in [24]), the computation time is about an order of magnitude slower than our proposed method, as reported in table 1
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 5.  Microscopy example cases. Top: This AFM image is severely affected by inhomogeneity compared to pattern contrast. As a result, classical CV segmentation fails. The proposed CV+XB model is able to capture some of the bias as such, and correctly segments some of the actual pattern. The central-square contour initialization, however, provokes an incorrect bias field estimate at early stages of the optimization, and leads to an incorrect local minimum, misclassifying the central portions. Middle: Starting from a near-optimal contour initialization of a single pattern element, the CV model still fails entirely. The proposed CV+XB model is not misled into incorrect minima, anymore, and successfully separates bias from actual pattern contrast. Bottom: The CV+XB model captures most of the inhomogeneity present in this AFM sample image and leads to reasonable segmentation of the diamond pattern. (Note: The inversion of foreground/background between CV and CV+XB models is arbitrary and triggered by domain size
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 6.  Further microscopy examples. Top: Seemingly "easy" AFM sample, actually affected by strong inhomogeneity at different scales. CV+XB yields a flattened structural image and correct pattern segmentation. Second set: Fluorescence microscopy image suffering from strong bias and artifacts, failing classical CV. CV+XB corrects bias and captures most artifacts. Third set: AFM sample with crack (white) and bend (darkening). The darkening is beyond recovery, but the crack is correctly identified as artifact. Bottom: Artifact detection collects bright spots and the dark line, resulting in correct stripe-pattern segmentation
Figure Options

Download full-size image

Download as PowerPoint slide

Table 1.  Algorithm convergence: number of iterations and required computer time. The algorithm is deterministic. Convergence is defined as the phase-field $u$ not changing during its update. Computation time is statistical depending on CPU scheduling; here, we report average numbers over 50 repetitions. The extra cost of artifact is negligible compared to basic CV, in particular since it may speed up convergence in appropriate images. Bias correction roughly doubles the computational load per iteration, which is an acceptable price for its benefits when appropriate
I M × N CV CV+X CV+B CV+XB
i [1] t [s] i [1] t [s] i [1] t [s] i [1] t [s]
1 78 × 119 11 0.02 20 0.03 11 0.05 11 0.05
2 75 × 79 14 0.02 16 0.02 65 0.24 66 0.25
3 96 × 127 22 0.06 45 0.12 19 0.11 35 0.21
4 110 × 111 15 0.04 30 0.08 71 0.41 100 0.58
5 131 × 103 33 0.11 33 0.12 30 0.26 36 0.32
6 124 × 184 11 0.05 18 0.07 55 0.45 26 0.23
Table Options

Download as excel

I M × N CV CV+X CV+B CV+XB
i [1] t [s] i [1] t [s] i [1] t [s] i [1] t [s]
1 78 × 119 11 0.02 20 0.03 11 0.05 11 0.05
2 75 × 79 14 0.02 16 0.02 65 0.24 66 0.25
3 96 × 127 22 0.06 45 0.12 19 0.11 35 0.21
4 110 × 111 15 0.04 30 0.08 71 0.41 100 0.58
5 131 × 103 33 0.11 33 0.12 30 0.26 36 0.32
6 124 × 184 11 0.05 18 0.07 55 0.45 26 0.23
[1]

Maika Goto, Kazunori Kuwana, Yasuhide Uegata, Shigetoshi Yazaki. A method how to determine parameters arising in a smoldering evolution equation by image segmentation for experiment's movies. Discrete and Continuous Dynamical Systems - S, 2021, 14 (3) : 881-891. doi: 10.3934/dcdss.2020233
[2]

Fan Jia, Xue-Cheng Tai, Jun Liu. Nonlocal regularized CNN for image segmentation. Inverse Problems and Imaging, 2020, 14 (5) : 891-911. doi: 10.3934/ipi.2020041
[3]

Yuan Wang, Zhi-Feng Pang, Yuping Duan, Ke Chen. Image retinex based on the nonconvex TV-type regularization. Inverse Problems and Imaging, 2021, 15 (6) : 1381-1407. doi: 10.3934/ipi.2020050
[4]

Ye Yuan, Yan Ren, Xiaodong Liu, Jing Wang. Approach to image segmentation based on interval neutrosophic set. Numerical Algebra, Control and Optimization, 2020, 10 (1) : 1-11. doi: 10.3934/naco.2019028
[5]

Matthew S. Keegan, Berta Sandberg, Tony F. Chan. A multiphase logic framework for multichannel image segmentation. Inverse Problems and Imaging, 2012, 6 (1) : 95-110. doi: 10.3934/ipi.2012.6.95
[6]

Shi Yan, Jun Liu, Haiyang Huang, Xue-Cheng Tai. A dual EM algorithm for TV regularized Gaussian mixture model in image segmentation. Inverse Problems and Imaging, 2019, 13 (3) : 653-677. doi: 10.3934/ipi.2019030
[7]

Jianping Zhang, Ke Chen, Bo Yu, Derek A. Gould. A local information based variational model for selective image segmentation. Inverse Problems and Imaging, 2014, 8 (1) : 293-320. doi: 10.3934/ipi.2014.8.293
[8]

Lu Tan, Ling Li, Senjian An, Zhenkuan Pan. Nonlinear diffusion based image segmentation using two fast algorithms. Mathematical Foundations of Computing, 2019, 2 (2) : 149-168. doi: 10.3934/mfc.2019011
[9]

Ruiliang Zhang, Xavier Bresson, Tony F. Chan, Xue-Cheng Tai. Four color theorem and convex relaxation for image segmentation with any number of regions. Inverse Problems and Imaging, 2013, 7 (3) : 1099-1113. doi: 10.3934/ipi.2013.7.1099
[10]

Balázs Kósa, Karol Mikula, Markjoe Olunna Uba, Antonia Weberling, Neophytos Christodoulou, Magdalena Zernicka-Goetz. 3D image segmentation supported by a point cloud. Discrete and Continuous Dynamical Systems - S, 2021, 14 (3) : 971-985. doi: 10.3934/dcdss.2020351
[11]

Jie Huang, Xiaoping Yang, Yunmei Chen. A fast algorithm for global minimization of maximum likelihood based on ultrasound image segmentation. Inverse Problems and Imaging, 2011, 5 (3) : 645-657. doi: 10.3934/ipi.2011.5.645
[12]

Liam Burrows, Weihong Guo, Ke Chen, Francesco Torella. Reproducible kernel Hilbert space based global and local image segmentation. Inverse Problems and Imaging, 2021, 15 (1) : 1-25. doi: 10.3934/ipi.2020048
[13]

Mehrdad Nojoumian, Douglas R. Stinson. On dealer-free dynamic threshold schemes. Advances in Mathematics of Communications, 2013, 7 (1) : 39-56. doi: 10.3934/amc.2013.7.39
[14]

Xinli Hu. Threshold dynamics for a Tuberculosis model with seasonality. Mathematical Biosciences & Engineering, 2012, 9 (1) : 111-122. doi: 10.3934/mbe.2012.9.111
[15]

Mehdi Bastani, Davod Khojasteh Salkuyeh. On the GSOR iteration method for image restoration. Numerical Algebra, Control and Optimization, 2021, 11 (1) : 27-43. doi: 10.3934/naco.2020013
[16]

Zhenguo Bai, Yicang Zhou. Threshold dynamics of a bacillary dysentery model with seasonal fluctuation. Discrete and Continuous Dynamical Systems - B, 2011, 15 (1) : 1-14. doi: 10.3934/dcdsb.2011.15.1
[17]

Jinhuo Luo, Jin Wang, Hao Wang. Seasonal forcing and exponential threshold incidence in cholera dynamics. Discrete and Continuous Dynamical Systems - B, 2017, 22 (6) : 2261-2290. doi: 10.3934/dcdsb.2017095
[18]

Baoli Shi, Zhi-Feng Pang, Jing Xu. Image segmentation based on the hybrid total variation model and the K-means clustering strategy. Inverse Problems and Imaging, 2016, 10 (3) : 807-828. doi: 10.3934/ipi.2016022
[19]

Yupeng Li, Wuchen Li, Guo Cao. Image segmentation via $ L_1 $ Monge-Kantorovich problem. Inverse Problems and Imaging, 2019, 13 (4) : 805-826. doi: 10.3934/ipi.2019037
[20]

Qianting Ma, Tieyong Zeng, Dexing Kong, Jianwei Zhang. Weighted area constraints-based breast lesion segmentation in ultrasound image analysis. Inverse Problems and Imaging, 2022, 16 (2) : 451-466. doi: 10.3934/ipi.2021057

2020 Impact Factor: 1.639

Tools

Metrics

  • PDF downloads (320)
  • HTML views (95)
  • Cited by (12)

Other articles
by authors

[Back to Top]

Copyright © 2022 American Institute of Mathematical Sciences | Privacy Policy