\`x^2+y_1+z_12^34\`
Advanced Search
Article Contents
Article Contents

Learnable Douglas-Rachford iteration and its applications in DOT imaging

  • * Corresponding author: Jiulong Liu

    * Corresponding author: Jiulong Liu 
Abstract Full Text(HTML) Figure(9) / Table(3) Related Papers Cited by
  • How to overcome the ill-posed nature of inverse problems is a pervasive problem in medical imaging. Most existing solutions are based on regularization techniques. This paper proposed a deep neural network (DNN) based image reconstruction method, the so-called DR-Net, that leverages the interpretability of existing regularization methods and adaptive modeling capacity of DNN. Motivated by a Douglas-Rachford fixed-point iteration for solving $ \ell_1 $-norm relating regularization model, the proposed DR-Net learns the prior of the solution via a U-Net based network, as well as other important regularization parameters. The DR-Net is applied to solve image reconstruction problem in diffusion optical tomography (DOT), a non-invasive imaging technique with many applications in medical imaging. The experiments on both simulated and experimental data showed that the proposed DNN based image reconstruction method significantly outperforms existing regularization methods.

    Mathematics Subject Classification: Primary: 94A08, 68U10; Secondary: 92B20.

    Citation:

    \begin{equation} \\ \end{equation}
  • 加载中
  • Figure 1.  A prototype time-resolved diffuse optical tomography system designed for optical imaging of human breast [24]

    Figure 2.  X-Net: architecture of components of DR-Net, including $ \mathcal{D_\theta} $, $ \Phi_\vartheta $ and $ \mathcal{D_\theta}^\top $. Note that the weights of $ \mathcal{D_\theta} $ and $ \mathcal{D_\theta}^\top $ can be shared with each other, or learned individually.

    Figure 3.  Phantom shapes for simulated data, each cubic container is of size $ 5mm \times 5mm\times 5mm $ ($ 1 \times 1 \times 1 $ voxel)

    Figure 4.  Reconstructed absorption coefficients from simulated measurements with phantom size of $ 10mm \times 10mm \times 5mm $ in depth of $ 30mm $

    Figure 5.  Reconstructed scattering coefficients from simulated measurements with phantom of $ 10mm \times 10mm \times 5mm $ in depth of $ 30mm $

    Figure 6.  Reconstructed absorption coefficients from experimental measurements $ \{I_{15}, I_{16}, I_{17}\} $ with phantom of $ 5 mm \times 10mm \times 5mm $ in depth of $ 25mm $

    Figure 7.  Reconstructed scattering coefficients from experimental measurements $ \{I_{15}, I_{16}, I_{17}\} $ with phantom of $ 5 mm \times 10mm \times 5mm $ in depth of $ 25mm $

    Figure 8.  Outputs of inversion blocks for absorption coefficients $ u_a^k $ of all stages in inference phase from simulated measurements with phantom size of $ 10mm \times 10mm \times 5mm $ in depth of $ 15mm $

    Figure 9.  Outputs of de-artifacting blocks for absorption coefficients $ \mathcal{D}_{\theta^\ast_k}^\top v_a^k $ of all stages in inference phase from simulated measurements with phantom size of $ 10mm \times 10mm \times 5mm $ in depth of $ 15mm $

    Table 1.  Experimental dataset ($ Q\neq \phi $)

    Depth(mm)515253545
    raw dataset$ T_1 = \{I_{i}\}_{1}^5 $$ T_2 = \{I_{i}\}_{6}^{13} $$ T_3 = \{I_{i}\}_{14}^{18} $$ T_4 = \{I_{i}\}_{15}^{19} $$ T_5 = \{I_{i}\}_{20}^{24} $
    Augmentation $ Q\subset T_1 $ $ Q\subset T_2 $ $ Q\subset T_3 $ $ Q\subset T_4 $ $ Q\subset T_5 $
    Data size31255313131
    PurposeTrainingTrainingTestingTrainingTraining
     | Show Table
    DownLoad: CSV

    Table 2.  CNR of reconstructed phantom in Fig. 4-7 from simulated data and experimental data

    Data Results Pixel Tikhonov TV Post-net Learned PD DR-Net
    sim $ u_a $ 1 1.04 1.33 13.53 83.65 125.94
    2 1.46 1.28 11.27 77.82 158.06
    3 1.58 1.42 16.02 66.74 100.64
    4 1.94 2.17 15.58 75.16 218.52
    $ u_s $ 1 1.48 2.33 6.06 84.81 187.18
    2 1.94 2.32 5.61 77.82 123.83
    3 1.18 1.17 4.83 66.74 51.55
    4 1.36 2.51 5.61 75.16 34.63
    exp $ u_a $ 1 1.75 1.12 3.14 0.92 5.09
    2 2.21 2.39 1.70 4.93 15.57
    $ u_s $ 1 1.38 0.64 1.16 0.96 7.09
    2 2.40 1.55 2.21 6.7607 18.98
     | Show Table
    DownLoad: CSV

    Table 3.  Averaged PSNR and SSIM of reconstructed images from simulated data and experimental data

    Data Results Measure Tikhonov TV Post-net Learned PD DR-Net
    sim $ u_a $ PSNR 31.68 31.91 38.22 39.46 $ {\mathbf 40.98} $
    SSIM 0.9182 0.9275 0.9759 0.9807 $ {\mathbf 0.9881} $
    $ u_s $ PSNR 31.29 32.04 34.56 $ {\mathbf 41.34} $ 38.03
    SSIM 0.9301 0.9401 0.9740 $ {\mathbf 0.9914 } $ 0.9898
    exp $ u_a $ PSNR 28.16 28.36 29.01 28.07 $ {\mathbf 29.33 } $
    SSIM 0.8612 0.8871 0.9441 0.9412 $ {\mathbf 0.9460 } $
    $ u_s $ PSNR 28.92 29.20 29.94 29.46 $ {\mathbf 31.13 } $
    SSIM 0.8870 0.9239 0.9587 0.9700 $ {\mathbf 0.9779} $
     | Show Table
    DownLoad: CSV
  • [1] J. Adler and O. Öktem, Learned primal-dual reconstruction, IEEE Transactions on Medical Imaging, 37 (2018), 1322-1332.  doi: 10.1109/TMI.2018.2799231.
    [2] J. Adler and O. Öktem, Solving ill-posed inverse problems using iterative deep neural networks, Inverse Problems, 33 (2017), 124007, 24 pp. doi: 10.1088/1361-6420/aa9581.
    [3] S. Boyd, N. Parikh, E. Chu, B. Peleato and J. Eckstein, Distributed optimization and statistical learning via the alternating direction method of multipliers, Foundations and Trends® in Machine Learning, 3 (2011), 1–122.
    [4] J.-F. CaiB. DongS. Osher and Z. Shen, Image restoration: Total variation, wavelet frames, and beyond, J. Amer. Math. Soc., 25 (2012), 1033-1089.  doi: 10.1090/S0894-0347-2012-00740-1.
    [5] J.-F. CaiH. JiZ. Shen and G.-B. Ye, Data-driven tight frame construction and image denoising, Appl. Comput. Harmon. Anal., 37 (2014), 89-105.  doi: 10.1016/j.acha.2013.10.001.
    [6] N. CaoA. Nehorai and M. Jacob, Image reconstruction for diffuse optical tomography using sparsity regularization and expectation-maximization algorithm, Optics Express, 15 (2007), 13695-13708.  doi: 10.1364/OE.15.013695.
    [7] K. Dabov, A. Foi, V. Katkovnik and K. Egiazarian, Image denoising with block-matching and 3d filtering, in Image Processing: Algorithms and Systems, Neural Networks, and Machine Learning, 6064, Proc. SPIE, 2006, 606414. doi: 10.1117/12.643267.
    [8] D. Davis and W. Yin, A three-operator splitting scheme and its optimization applications, Set-Valued Var. Anal., 25 (2017), 829-858.  doi: 10.1007/s11228-017-0421-z.
    [9] T. Goldstein and S. Osher, The split Bregman method for $L1$-regularized problems, SIAM J. Imaging Sci., 2 (2009), 323-343.  doi: 10.1137/080725891.
    [10] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, et al., Generative adversarial nets, in Advances in Neural Information Processing Systems, 2014, 2672–2680.
    [11] I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin and A. Courville, Improved training of Wasserstein GANs, preprint, arXiv: 1704.00028.
    [12] K. HammernikT. KlatzerE. KoblerM. P. RechtD. K. SodicksonT. Pock and F. Knoll, Learning a variational network for reconstruction of accelerated MRI data, Magnetic Resonance in Medicine, 79 (2018), 3055-3071.  doi: 10.1007/978-3-319-66709-6.
    [13] K. H. JinM. T. McCannE. Froustey and M. Unser, Deep convolutional neural network for inverse problems in imaging, IEEE Trans. Image Process., 26 (2017), 4509-4522.  doi: 10.1109/TIP.2017.2713099.
    [14] E. Kang, J. Min and J. C. Ye, A deep convolutional neural network using directional wavelets for low-dose x-ray CT reconstruction, Medical Physics, 44 (2017), e360–e375. doi: 10.1002/mp.12344.
    [15] V. KolehmainenM. VauhkonenJ. P. Kaipio and S. R. Arridge, Recovery of piecewise constant coefficients in optical diffusion tomography, Optics Express, 7 (2000), 468-480.  doi: 10.1364/OE.7.000468.
    [16] C. Ledig, L. Theis, F. Huszár, J. Caballero, A. Cunningham, A. Acosta, et al., Photo-realistic single image super-resolution using a generative adversarial network, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2017), 105–114. doi: 10.1109/CVPR.2017.19.
    [17] P.-L. Lions and B. Mercier, Splitting algorithms for the sum of two nonlinear operators, SIAM J. Numer. Anal., 16 (1979), 964-979.  doi: 10.1137/0716071.
    [18] J. Liu, Y. Hu, J. Yang, Y. Chen, H. Shu, L. Luo, et al., 3d feature constrained reconstruction for low-dose CT imaging, IEEE Transactions on Circuits and Systems for Video Technology, 28 (2018), 1232–1247. doi: 10.1109/TCSVT.2016.2643009.
    [19] J. Liu, A. I. Aviles-Rivero, H. Ji and C.-B. Schönlieb, Rethinking medical image reconstruction via shape prior, going deeper and faster: Deep joint indirect registration and reconstruction, preprint, arXiv: 1912.07648. doi: 10.1016/j.mathsocsci.2015.03.001.
    [20] J. LiuH. DingS. MolloiX. Zhang and H. Gao, TICMR: Total image constrained material reconstruction via nonlocal total variation regularization for spectral CT, IEEE Transactions on Medical Imaging, 35 (2016), 2578-2586.  doi: 10.1109/TMI.2016.2587661.
    [21] J. Liu, T. Kuang and X. Zhang, Image reconstruction by splitting deep learning regularization from iterative inversion, in Medical Image Computing and Computer Assisted Intervention, 11070, Lecture Notes in Computer Science, Springer, Cham, 2018,224–231. doi: 10.1007/978-3-030-00928-1_26.
    [22] T. LiuM. Gong and D. Tao, Large-cone nonnegative matrix factorization, IEEE Trans. Neural Netw. Learn. Syst., 28 (2017), 2129-2142.  doi: 10.1109/tnnls.2016.2514360.
    [23] T. Meinhardt, M. Moller, C. Hazirbas and D. Cremers, Learning proximal operators: Using denoising networks for regularizing inverse imaging problems, in IEEE International Conference on Computer Vision (ICCV), 2017, 1781–1790. doi: 10.1109/ICCV.2017.198.
    [24] W. Mo and N. Chen, Design of an advanced time-domain diffuse optical tomography system, IEEE Journal of Selected Topics in Quantum Electronics, 16 (2010), 581-587. 
    [25] S. Nowozin, B. Cseke and R. Tomioka, f-GAN: Training generative neural samplers using variational divergence minimization, in Advances in Neural Information Processing Systems, 2016,271–279.
    [26] O. Ronneberger, P. Fischer and T. Brox, U-net: Convolutional networks for biomedical image segmentation, in International Conference on Medical Image Computing and Computer-Assisted Intervention, 9351, Lecture Notes in Computer Science, Springer, Cham, 2015,234–241. doi: 10.1007/978-3-319-24574-4_28.
    [27] L. I. RudinS. Osher and E. Fatemi, Nonlinear total variation based noise removal algorithms, Phys. D, 60 (1992), 259-268.  doi: 10.1016/0167-2789(92)90242-F.
    [28] J. Sun, H. Li, Z. Xu and Y. Yang, Deep ADMM-Net for compressive sensing MRI, in Advances in Neural Information Processing Systems, 2016, 10–18.
    [29] A. N. Tihonov, Solution of incorrectly formulated problems and the regularization method, Soviet Math. Dokl., 4 (1963), 1035-1038. 
    [30] P. Vincent, H. Larochelle, Y. Bengio and P.-A. Manzagol, Extracting and composing robust features with denoising autoencoders, in Proceedings of the 25th International Conference on Machine learning, ACM, 2008, 1096–1103. doi: 10.1145/1390156.1390294.
    [31] J. Xie, L. Xu and E. Chen, Image denoising and inpainting with deep neural networks, in Advances in Neural Information Processing Systems, 2012,341–349.
    [32] X. YangR. KwittM. Styner and M. Niethammer, Quicksilver: Fast predictive image registration– A deep learning approach, NeuroImage, 158 (2017), 378-396.  doi: 10.1016/j.neuroimage.2017.07.008.
    [33] J. Yoo, S. Sabir, D. Heo, K. H. Kim, A. Wahab, Y. Choi, et al., Deep learning diffuse optical tomography, IEEE Transactions on Medical Imaging, 39 (2020), 877–887. doi: 10.1109/TMI.2019.2936522.
  • 加载中

Figures(9)

Tables(3)

SHARE

Article Metrics

HTML views(1435) PDF downloads(443) Cited by(0)

Access History

Other Articles By Authors

Catalog

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return