# American Institute of Mathematical Sciences

December  2020, 7(2): 469-487. doi: 10.3934/jcd.2020019

## Time-resolved denoising using model order reduction, dynamic mode decomposition, and kalman filter and smoother

 1 Department of Mechanical Engineering, University of Wisconsin-Milwaukee, Milwaukee, WI, USA 2 Department of Electrical and Computer Engineering, New York Institute of Technology, Long Island, NY, USA 3 Departments of Neurosurgery, Medical College of Wisconsin, Milwaukee, WI, USA

* Corresponding author: Mojtaba F. Fathi

Received  September 2019 Published  July 2020

In this research, we investigate the application of Dynamic Mode Decomposition combined with Kalman Filtering, Smoothing, and Wavelet Denoising (DMD-KF-W) for denoising time-resolved data. We also compare the performance of this technique with state-of-the-art denoising methods such as Total Variation Diminishing (TV) and Divergence-Free Wavelets (DFW), when applicable. Dynamic Mode Decomposition (DMD) is a data-driven method for finding the spatio-temporal structures in time series data. In this research, we use an autoregressive linear model resulting from applying DMD to the time-resolved data as a predictor in a Kalman Filtering-Smoothing framework for the purpose of denoising. The DMD-KF-W method is parameter-free and runs autonomously. Tests on numerical phantoms show lower error metrics when compared to TV and DFW, when applicable. In addition, DMD-KF-W runs an order of magnitude faster than DFW and TV. In the case of synthetic datasets, where the noise-free datasets were available, our method was shown to perform better than TV and DFW methods (when applicable) in terms of the defined error metric.

Citation: Mojtaba F. Fathi, Ahmadreza Baghaie, Ali Bakhshinejad, Raphael H. Sacho, Roshan M. D'Souza. Time-resolved denoising using model order reduction, dynamic mode decomposition, and kalman filter and smoother. Journal of Computational Dynamics, 2020, 7 (2) : 469-487. doi: 10.3934/jcd.2020019
##### References:
 [1] E. Bostan, S. Lefkimmiatis, O. Vardoulis, N. Stergiopulos and M. Unser, Improved variational denoising of flow fields with application to phase-contrast MRI data, IEEE Signal Processing Letters, 22 (2015), 762-766.  doi: 10.1109/LSP.2014.2369212.  Google Scholar [2] K. K. Chen, J. H. Tu and C. W. Rowley, Variants of dynamic mode decomposition: Boundary condition, Koopman, and Fourier analyses, J. Nonlinear Sci., 22 (2012), 887-915.  doi: 10.1007/s00332-012-9130-9.  Google Scholar [3] Y. Chen, A. Wiesel and A. O. Hero, Shrinkage Estimation of High Dimensional Covariance Matrices, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing, 2937–2940. Google Scholar [4] R. R. Coifman and D. L. Donoho, Translation-invariant de-noising, in Wavelets and Statistics, Springer, (1995), 125–150. Google Scholar [5] J. L. Crassidis and J. L. Junkins, Optimal Estimation of Dynamic Systems, Chapman and Hall/CRC, 2004. doi: 10.1201/9780203509128.  Google Scholar [6] S. T. M. Dawson, M. S. Hemati, M. O. Williams and C. W. Rowley, Characterizing and correcting for the effect of sensor noise in the dynamic mode decomposition, Experiments in Fluids, 57 (2016), 1-19.  doi: 10.1007/s00348-016-2127-7.  Google Scholar [7] M. F. Fathi, A. Bakhshinejad, A. Baghaie and R. M. D'Souza, Dynamic denoising and gappy data reconstruction based on dynamic mode decomposition and discrete cosine transform, Applied Sciences, 8 (2018), 1515. doi: 10.3390/app8091515.  Google Scholar [8] M. F. Fathi, A. Bakhshinejad, A. Baghaie, D. Saloner, R. H. Sacho, V. L. Rayz and R. M. D'Souza, Denoising and spatial resolution enhancement of 4D flow MRI using proper orthogonal decomposition and Lasso regularization, Computerized Medical Imaging and Graphics, 70 (2018), 165-172.  doi: 10.1016/j.compmedimag.2018.07.003.  Google Scholar [9] M. Gavish and D. L. Donoho, The optimal hard threshold for singular values is $4/\sqrt 3$, IEEE Trans. Inform. Theory, 60 (2014), 5040-5053.  doi: 10.1109/TIT.2014.2323359.  Google Scholar [10] A. Gelb, Applied Optimal Estimation, MIT press, 1974.   Google Scholar [11] G. V. Iungo, C. Santoni-Ortiz, M. Abkar, F. Porté-Agel, M. A. Rotea and S. Leonardi, Data-driven reduced order model for prediction of wind turbine wakes, Journal of Physics: Conference Series, 625. doi: 10.1088/1742-6596/625/1/012009.  Google Scholar [12] K. M. Johnson and M. Markl, Improved SNR in phase contrast velocimetry with five-point balanced flow encoding, Magnetic Resonance in Medicine, 63 (2010), 349-355.  doi: 10.1002/mrm.22202.  Google Scholar [13] R. E. Kalman, A new approach to linear filtering and prediction problems, Trans. ASME Ser. D. J. Basic Engrg., 82 (1960), 35-45.  doi: 10.1115/1.3662552.  Google Scholar [14] J. N. Kutz, S. L. Brunton, B. W. Brunton and J. L. Proctor, Dynamic Mode Decomposition: Data-Driven Modeling of Complex Systems, SIAM, 2016. doi: 10.1137/1.9781611974508.  Google Scholar [15] T. Nonomura, H. Shibata and R. Takaki, Dynamic mode decomposition using a kalman filter for parameter estimation, AIP Advances, 8 (2018), 105106. doi: 10.1063/1.5031816.  Google Scholar [16] T. Nonomura, H. Shibata and R. Takaki, Extended-kalman-filter-based dynamic mode decomposition for simultaneous system identification and denoising, PloS one, 14, e0209836. doi: 10.1371/journal.pone.0209836.  Google Scholar [17] F. Ong, M. Uecker, U. Tariq, A. Hsiao, M. T. Alley, S. S. Vasanawala and M. Lustig, Robust 4D flow denoising using divergence-free wavelet transform, Magnetic Resonance in Medicine, 73 (2015), 828-842.  doi: 10.1002/mrm.25176.  Google Scholar [18] H. E. Rauch, F. Tung and C. T. Striebel, Maximum likelihood estimates of linear dynamic systems, AIAA J., 3 (1965), 1445-1450.  doi: 10.2514/3.3166.  Google Scholar [19] C. W. Rowley, I. Mezić, S. Bagheri, P. Schlatter and D. S. Henningson, Spectral analysis of nonlinear flows, J. Fluid Mech., 641 (2009), 115-127.  doi: 10.1017/S0022112009992059.  Google Scholar [20] P. J. Schmid, Dynamic mode decomposition of numerical and experimental data, J. Fluid Mech., 656 (2010), 5-28.  doi: 10.1017/S0022112010001217.  Google Scholar [21] P. J. Schmid and J. Sesterhenn, Dynamic mode decomposition of numerical and experimental data, Bulletin of the American Physical Society, 53 (2008), 2008. Google Scholar [22] J. H. Tu, C. W. Rowley, D. M. Luchtenburg, S. L. Brunton and J. N. Kutz, On dynamic mode decomposition: Theory and applications, Journal of Computational Dynamics, 1 (2014), 391-421.  doi: 10.3934/jcd.2014.1.391.  Google Scholar [23] E. Wan, Sigma-point filters: An overview with applications to integrated navigation and vision assisted control, in 2006 IEEE Nonlinear Statistical Signal Processing Workshop, (2006), 201–202. doi: 10.1109/NSSPW.2006.4378854.  Google Scholar [24] K. Willcox and J. Peraire, Balanced model reduction via the proper orthogonal decomposition, AIAA Journal, 40 (2002), 2323-2330.  doi: 10.2514/6.2001-2611.  Google Scholar [25] M. O. Williams, C. W. Rowley and I. G. Kevrekidis, A kernel-based method for data-driven Koopman spectral analysis, J. Comput. Dyn., 2 (2015), 247–265. arXiv: 1411.2260. doi: 10.3934/jcd.2015005.  Google Scholar [26] M. O. Williams, I. G. Kevrekidis and C. W. Rowley, A data-driven approximation of the Koopman operator: Extending dynamic mode decomposition, J. Nonlinear Sci., 25 (2015), 1307-1346.  doi: 10.1007/s00332-015-9258-5.  Google Scholar [27] M. J. Zimoń, J. M. Reese and D. R. Emerson, A novel coupling of noise reduction algorithms for particle flow simulations, J. Comput. Phys., 321 (2016), 169-190.  doi: 10.1016/j.jcp.2016.05.049.  Google Scholar

show all references

##### References:
 [1] E. Bostan, S. Lefkimmiatis, O. Vardoulis, N. Stergiopulos and M. Unser, Improved variational denoising of flow fields with application to phase-contrast MRI data, IEEE Signal Processing Letters, 22 (2015), 762-766.  doi: 10.1109/LSP.2014.2369212.  Google Scholar [2] K. K. Chen, J. H. Tu and C. W. Rowley, Variants of dynamic mode decomposition: Boundary condition, Koopman, and Fourier analyses, J. Nonlinear Sci., 22 (2012), 887-915.  doi: 10.1007/s00332-012-9130-9.  Google Scholar [3] Y. Chen, A. Wiesel and A. O. Hero, Shrinkage Estimation of High Dimensional Covariance Matrices, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing, 2937–2940. Google Scholar [4] R. R. Coifman and D. L. Donoho, Translation-invariant de-noising, in Wavelets and Statistics, Springer, (1995), 125–150. Google Scholar [5] J. L. Crassidis and J. L. Junkins, Optimal Estimation of Dynamic Systems, Chapman and Hall/CRC, 2004. doi: 10.1201/9780203509128.  Google Scholar [6] S. T. M. Dawson, M. S. Hemati, M. O. Williams and C. W. Rowley, Characterizing and correcting for the effect of sensor noise in the dynamic mode decomposition, Experiments in Fluids, 57 (2016), 1-19.  doi: 10.1007/s00348-016-2127-7.  Google Scholar [7] M. F. Fathi, A. Bakhshinejad, A. Baghaie and R. M. D'Souza, Dynamic denoising and gappy data reconstruction based on dynamic mode decomposition and discrete cosine transform, Applied Sciences, 8 (2018), 1515. doi: 10.3390/app8091515.  Google Scholar [8] M. F. Fathi, A. Bakhshinejad, A. Baghaie, D. Saloner, R. H. Sacho, V. L. Rayz and R. M. D'Souza, Denoising and spatial resolution enhancement of 4D flow MRI using proper orthogonal decomposition and Lasso regularization, Computerized Medical Imaging and Graphics, 70 (2018), 165-172.  doi: 10.1016/j.compmedimag.2018.07.003.  Google Scholar [9] M. Gavish and D. L. Donoho, The optimal hard threshold for singular values is $4/\sqrt 3$, IEEE Trans. Inform. Theory, 60 (2014), 5040-5053.  doi: 10.1109/TIT.2014.2323359.  Google Scholar [10] A. Gelb, Applied Optimal Estimation, MIT press, 1974.   Google Scholar [11] G. V. Iungo, C. Santoni-Ortiz, M. Abkar, F. Porté-Agel, M. A. Rotea and S. Leonardi, Data-driven reduced order model for prediction of wind turbine wakes, Journal of Physics: Conference Series, 625. doi: 10.1088/1742-6596/625/1/012009.  Google Scholar [12] K. M. Johnson and M. Markl, Improved SNR in phase contrast velocimetry with five-point balanced flow encoding, Magnetic Resonance in Medicine, 63 (2010), 349-355.  doi: 10.1002/mrm.22202.  Google Scholar [13] R. E. Kalman, A new approach to linear filtering and prediction problems, Trans. ASME Ser. D. J. Basic Engrg., 82 (1960), 35-45.  doi: 10.1115/1.3662552.  Google Scholar [14] J. N. Kutz, S. L. Brunton, B. W. Brunton and J. L. Proctor, Dynamic Mode Decomposition: Data-Driven Modeling of Complex Systems, SIAM, 2016. doi: 10.1137/1.9781611974508.  Google Scholar [15] T. Nonomura, H. Shibata and R. Takaki, Dynamic mode decomposition using a kalman filter for parameter estimation, AIP Advances, 8 (2018), 105106. doi: 10.1063/1.5031816.  Google Scholar [16] T. Nonomura, H. Shibata and R. Takaki, Extended-kalman-filter-based dynamic mode decomposition for simultaneous system identification and denoising, PloS one, 14, e0209836. doi: 10.1371/journal.pone.0209836.  Google Scholar [17] F. Ong, M. Uecker, U. Tariq, A. Hsiao, M. T. Alley, S. S. Vasanawala and M. Lustig, Robust 4D flow denoising using divergence-free wavelet transform, Magnetic Resonance in Medicine, 73 (2015), 828-842.  doi: 10.1002/mrm.25176.  Google Scholar [18] H. E. Rauch, F. Tung and C. T. Striebel, Maximum likelihood estimates of linear dynamic systems, AIAA J., 3 (1965), 1445-1450.  doi: 10.2514/3.3166.  Google Scholar [19] C. W. Rowley, I. Mezić, S. Bagheri, P. Schlatter and D. S. Henningson, Spectral analysis of nonlinear flows, J. Fluid Mech., 641 (2009), 115-127.  doi: 10.1017/S0022112009992059.  Google Scholar [20] P. J. Schmid, Dynamic mode decomposition of numerical and experimental data, J. Fluid Mech., 656 (2010), 5-28.  doi: 10.1017/S0022112010001217.  Google Scholar [21] P. J. Schmid and J. Sesterhenn, Dynamic mode decomposition of numerical and experimental data, Bulletin of the American Physical Society, 53 (2008), 2008. Google Scholar [22] J. H. Tu, C. W. Rowley, D. M. Luchtenburg, S. L. Brunton and J. N. Kutz, On dynamic mode decomposition: Theory and applications, Journal of Computational Dynamics, 1 (2014), 391-421.  doi: 10.3934/jcd.2014.1.391.  Google Scholar [23] E. Wan, Sigma-point filters: An overview with applications to integrated navigation and vision assisted control, in 2006 IEEE Nonlinear Statistical Signal Processing Workshop, (2006), 201–202. doi: 10.1109/NSSPW.2006.4378854.  Google Scholar [24] K. Willcox and J. Peraire, Balanced model reduction via the proper orthogonal decomposition, AIAA Journal, 40 (2002), 2323-2330.  doi: 10.2514/6.2001-2611.  Google Scholar [25] M. O. Williams, C. W. Rowley and I. G. Kevrekidis, A kernel-based method for data-driven Koopman spectral analysis, J. Comput. Dyn., 2 (2015), 247–265. arXiv: 1411.2260. doi: 10.3934/jcd.2015005.  Google Scholar [26] M. O. Williams, I. G. Kevrekidis and C. W. Rowley, A data-driven approximation of the Koopman operator: Extending dynamic mode decomposition, J. Nonlinear Sci., 25 (2015), 1307-1346.  doi: 10.1007/s00332-015-9258-5.  Google Scholar [27] M. J. Zimoń, J. M. Reese and D. R. Emerson, A novel coupling of noise reduction algorithms for particle flow simulations, J. Comput. Phys., 321 (2016), 169-190.  doi: 10.1016/j.jcp.2016.05.049.  Google Scholar
The results of the unforced Duffing equation dataset. On the left, the noise-free, noisy, and reconstructed datasets are shown. The numbers at the top of the columns show the corresponding snapshot numbers. The two top rows show the noise-free (reference) and noisy datasets, respectively. The reconstructions of TV and DMD-KF-W methods are shown as indicated. The numbers inside the parenthesis are the overall RMSE values, where the lower is better. On the right, per-snapshot (solid lines) and overall (dotted) RMSE values of the noisy and the reconstructed datasets are presented
The results of second test case (wake behind a cylinder at Re = 100). On the left, the reference, noisy, and reconstructed datasets are shown. The images show the snapshot number 21. The bottom row shows the reconstruction errors with the overall RMSE values, where the lower is better. On the right, per-snapshot (solid lines) and overall (dotted) RMSE values of the noisy and the reconstructed datasets are presented
The effects of the tuning parameters of TV and DFW on the reconstruction error for the second test dataset. For TV (on left), the effect of the weighting parameter of the regularization term is shown where $\lambda = 0.07$ resulted in the best reconstruction. For DFW (on right), the effects of the smallest wavelet level size and the number of cycles are shown. In this case, 5 cycles with the smallest wavelet level size of 10 resulted in the best reconstruction
The effects of tuning parameters of TV and DFW on the reconstruction error for the synthetic 4D-Flow MRI dataset. For TV (on left), the effect of the weighting parameter of the regularization term is shown where $\lambda = 0.03$ resulted in the best reconstruction. For DFW (on right), the effects of the smallest wavelet level size and the number of cycles are shown. In this case, 4 cycles with the smallest wavelet level size of 12 resulted in the best reconstruction
The results for the synthetic 4D-Flow MRI dataset. The synthetic data resembles a cardiac cycle. (a) The top row shows sample slices where at each point, the norm of the velocity is shown. The bottom row shows the norm of difference between the datasets and the noise-free reference. The bottom row figures are titled with the corresponding total RMSE values. All areas outside the geometry mask are zeroed before RMSE values are calculated. (b) Per-snapshot (solid lines) and total (dotted) RMSE values of the noisy and the reconstructed datasets
The results for the two in-vivo 4D-Flow MRI datasets. For both datasets, the binary mask is used to exclude all points outside the vessel. The spatial resolutions of datasets (a) and (b) are $30 \times 40 \times 103$ and $22 \times 30 \times 48$, respectively
The effect of changing the type of Daubechies wavelet used for denoising on the RMSE value
Evaluating the effectiveness of wavelet denoising step of the proposed method in improving the RMSE values for the synthetic test cases. The horizontal axis shows the test case number. The vertical axis shows the ratio of the RMSE of reconstruction to the RMSE of the noisy input dataset in percent. A lower ratio corresponds to better denoising
 [1] Abdelghafour Atlas, Mostafa Bendahmane, Fahd Karami, Driss Meskine, Omar Oubbih. A nonlinear fractional reaction-diffusion system applied to image denoising and decomposition. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020321 [2] Dan Zhu, Rosemary A. Renaut, Hongwei Li, Tianyou Liu. Fast non-convex low-rank matrix decomposition for separation of potential field data using minimal memory. Inverse Problems & Imaging, , () : -. doi: 10.3934/ipi.2020076 [3] Shuang Chen, Jinqiao Duan, Ji Li. Effective reduction of a three-dimensional circadian oscillator model. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020349 [4] Chao Xing, Jiaojiao Pan, Hong Luo. Stability and dynamic transition of a toxin-producing phytoplankton-zooplankton model with additional food. Communications on Pure & Applied Analysis, , () : -. doi: 10.3934/cpaa.2020275 [5] Håkon Hoel, Gaukhar Shaimerdenova, Raúl Tempone. Multilevel Ensemble Kalman Filtering based on a sample average of independent EnKF estimators. Foundations of Data Science, 2020  doi: 10.3934/fods.2020017 [6] Zhiyan Ding, Qin Li, Jianfeng Lu. Ensemble Kalman Inversion for nonlinear problems: Weights, consistency, and variance bounds. Foundations of Data Science, 2020  doi: 10.3934/fods.2020018 [7] Ebraheem O. Alzahrani, Muhammad Altaf Khan. Androgen driven evolutionary population dynamics in prostate cancer growth. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020426 [8] Zonghong Cao, Jie Min. Selection and impact of decision mode of encroachment and retail service in a dual-channel supply chain. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2020167 [9] Bahaaeldin Abdalla, Thabet Abdeljawad. Oscillation criteria for kernel function dependent fractional dynamic equations. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020443 [10] Xin Guo, Lei Shi. Preface of the special issue on analysis in data science: Methods and applications. Mathematical Foundations of Computing, 2020, 3 (4) : i-ii. doi: 10.3934/mfc.2020026 [11] Hoang The Tuan. On the asymptotic behavior of solutions to time-fractional elliptic equations driven by a multiplicative white noise. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020318 [12] Annegret Glitzky, Matthias Liero, Grigor Nika. Dimension reduction of thermistor models for large-area organic light-emitting diodes. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020460 [13] Marion Darbas, Jérémy Heleine, Stephanie Lohrengel. Numerical resolution by the quasi-reversibility method of a data completion problem for Maxwell's equations. Inverse Problems & Imaging, 2020, 14 (6) : 1107-1133. doi: 10.3934/ipi.2020056 [14] Haixiang Yao, Ping Chen, Miao Zhang, Xun Li. Dynamic discrete-time portfolio selection for defined contribution pension funds with inflation risk. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2020166 [15] Manil T. Mohan. First order necessary conditions of optimality for the two dimensional tidal dynamics system. Mathematical Control & Related Fields, 2020  doi: 10.3934/mcrf.2020045 [16] Weisong Dong, Chang Li. Second order estimates for complex Hessian equations on Hermitian manifolds. Discrete & Continuous Dynamical Systems - A, 2020  doi: 10.3934/dcds.2020377 [17] Laurence Cherfils, Stefania Gatti, Alain Miranville, Rémy Guillevin. Analysis of a model for tumor growth and lactate exchanges in a glioma. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020457 [18] Laurent Di Menza, Virginie Joanne-Fabre. An age group model for the study of a population of trees. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020464 [19] Kihoon Seong. Low regularity a priori estimates for the fourth order cubic nonlinear Schrödinger equation. Communications on Pure & Applied Analysis, 2020, 19 (12) : 5437-5473. doi: 10.3934/cpaa.2020247 [20] Shasha Hu, Yihong Xu, Yuhan Zhang. Second-Order characterizations for set-valued equilibrium problems with variable ordering structures. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2020164

Impact Factor:

## Tools

Article outline

Figures and Tables