
-
Previous Article
A $ {BMAP/BMSP/1} $ queue with Markov dependent arrival and Markov dependent service batches
- JIMO Home
- This Issue
-
Next Article
A better dominance relation and heuristics for Two-Machine No-Wait Flowshops with Maximum Lateness Performance Measure
Principal component analysis with drop rank covariance matrix
School of Information Engineering, Guangdong University of Technology, Guangzhou, 510006, China |
This paper considers the principal component analysis when the covariance matrix of the input vectors drops rank. This case sometimes happens when the total number of the input vectors is very limited. First, it is found that the eigen decomposition of the covariance matrix is not uniquely defined. This implies that different transform matrices could be obtained for performing the principal component analysis. Hence, the generalized form of the eigen decomposition of the covariance matrix is given. Also, it is found that the matrix with its columns being the eigenvectors of the covariance matrix is not necessary to be unitary. This implies that the transform for performing the principal component analysis may not be energy preserved. To address this issue, the necessary and sufficient condition for the matrix with its columns being the eigenvectors of the covariance matrix to be unitary is derived. Moreover, since the design of the unitary transform matrix for performing the principal component analysis is usually formulated as an optimization problem, the necessary and sufficient condition for the first order derivative of the Lagrange function to be equal to the zero vector is derived. In fact, the unitary matrix with its columns being the eigenvectors of the covariance matrix is only a particular case of the condition. Furthermore, the necessary and sufficient condition for the second order derivative of the Lagrange function to be a positive definite function is derived. It is found that the unitary matrix with its columns being the eigenvectors of the covariance matrix does not satisfy this condition. Computer numerical simulation results are given to valid the results.
References:
[1] |
J. Dehaene, M. Moonen and J. Vandewalle, An improved stochastic gradient algorithm for principal component analysis and subspace tracking, IEEE Transactions on Signal Processing, 45 (1997), 2582-2586. Google Scholar |
[2] |
C. L. Fancourt and J. C. Principe,
Competitive principal component analysis for locally stationary time series, IEEE Transactions on Signal Processing, 46 (1998), 3068-3081.
doi: 10.1109/78.726819. |
[3] |
J. B. O. S. Filho and P. S. R. Diniz,
A fixed-point online kernel principal component extraction algorithm, IEEE Transactions on Signal Processing, 65 (2017), 6244-6259.
doi: 10.1109/TSP.2017.2750119. |
[4] |
I. A. Guimarães and A. C. Neto,
Estimation in polytomous logistic model: Comparison of methods, Journal of Industrial and Management Optimization, Journal of Industrial and Management Optimization, 5 (2009), 239-252.
doi: 10.3934/jimo.2009.5.239. |
[5] |
R. He, B. G. Hu, W. S. Zheng and X. W. Kong,
Robust principal component analysis based on maximum correntropy criterion, IEEE Transactions on Image Processing, 20 (2011), 1485-1494.
doi: 10.1109/TIP.2010.2103949. |
[6] |
S. M. Huang and J. F. Yang,
Improved principal component regression for face recognition under illumination variations, IEEE Signal Processing Letters, 19 (2012), 179-182.
doi: 10.1109/LSP.2012.2185492. |
[7] |
J. Kang, X. Lin and G. Yang, Research of multi-scale PCA algorithm for face recognition, International Conference on Information and Communications Technologies, ICT, (2015), 1–5. Google Scholar |
[8] |
M. S. Kang, J. H. Bae, B. S. Kang and K. T. Kim,
ISAR cross-range scaling using iterative processing via principal component analysis and bisection algorithm, IEEE Transactions on Signal Processing, 64 (2016), 3909-3918.
doi: 10.1109/TSP.2016.2552511. |
[9] |
S. Karamizadeh, S. M. Abdullah, A. A. Manaf and M. Zamani,
An overview of principal component analysis, Journal of Signal and Information Processing, 4 (2013), 173-175.
doi: 10.4236/jsip.2013.43B031. |
[10] |
H. C. Liu, S. T. Li and L. Y. Fang, Robust object tracking based on principal component analysis and local sparse representation, IEEE Transactions on Instrumentation and Measurement, 64 (2015), 2863-2875. Google Scholar |
[11] |
Z. Li, Q. L. Ye, Y. T. Guo, Z. K. Tian, B. W. K. Ling and R. W. K. Lam, Wearable non-invasive blood glucose estimation via empirical mode decomposition based hierarchical multiresolution analysis and random forest, 23rd IEEE International Conference on Digital Signal Processing, DSP, (2018), 1–5.
doi: 10.1109/ICDSP.2018.8631545. |
[12] |
A. Moradi, J. Razmi, R. Babazadeh and A. Sabbaghnia,
An integrated Principal Component Analysis and multi-objective mathematical programming approach to agile supply chain network design under uncertainty, Journal of Industrial and Management Optimization, 15 (2019), 855-879.
doi: 10.3934/jimo.2018074. |
[13] |
E. M. Moreno, Non-invasive estimate of blood glucose and blood pressure from a photoplethysmography by means of machine learning techniques, Artificial Intelligence in Medicine, 53 (2011), 127-138. Google Scholar |
[14] |
P. P. Markopulos, G. N. Karystinos and D. A. Pados,
Optimal algorithms for L1-subpace signal processing, IEEE Transactions on Signal Processing, 62 (2014), 5046-5058.
doi: 10.1109/TSP.2014.2338077. |
[15] |
O. Ozgonenel and T. Yalcin, Principal component analysis (PCA) based neural network for motor protection, 10th IET International Conference on Developments in Power System Protection, DPSP, (2010), 1–5.
doi: 10.1049/cp.2010.0252. |
[16] |
S. Ouyang and Z. Bao,
Fast principal component extraction by a weighted information criterion, IEEE Transactions on Signal Processing, 50 (2002), 1994-2002.
doi: 10.1109/TSP.2002.800395. |
[17] |
Y. M. Ren, L. Liao and S. J. Maybank,
Hyperspectral image spectral-spatial feature extraction via tensor principal component analysis, IEEE Geoscience and Remote Sensing Letters, 14 (2017), 1431-1435.
doi: 10.1109/LGRS.2017.2686878. |
[18] |
C. E. Thomaz and G. A. Giraldi,
A new ranking method for principal components analysis and its application to face image analysis, Image and Vision Computing, 28 (2009), 902-913.
doi: 10.1016/j.imavis.2009.11.005. |
[19] |
G. Tang and A. Nehorai,
Constrained Cramér-Rao bound on robust principal component analysis, IEEE Transactions on Signal Processing, 59 (2011), 5070-5076.
doi: 10.1109/TSP.2011.2161984. |
[20] |
X. C Xiu, Y. Yang, W.Q. Liu, L.C. Kong and M.J. Shang,
An improved total variation regularized RPCA for moving object detection with dynamic background, Journal of Industrial and Management Optimization, 13 (2017), 1-14.
doi: 10.3934/jimo.2019024. |
[21] |
S. Y. Yi, Z. H. Lai, Z. Y. He, Y. Cheung and Y. Liu,
Joint sparse principal component analysis, Pattern Recognition, 61 (2016), 524-536.
doi: 10.1016/j.patcog.2016.08.025. |
[22] |
A. Zare, A. Ozdemir, M. A. Iwen and S. Aviyente,
Extension of PCA to higher order data structures: An introduction to tensors, tensor decompositions, and tensor PCA, Proceedings of the IEEE, 106 (2018), 1341-1358.
doi: 10.1109/JPROC.2018.2848209. |
[23] |
H. Zou and L. Xue,
A selective overview of sparse principal component analysis, Proceedings of the IEEE, 106 (2018), 1311-1320.
doi: 10.1109/JPROC.2018.2846588. |
show all references
References:
[1] |
J. Dehaene, M. Moonen and J. Vandewalle, An improved stochastic gradient algorithm for principal component analysis and subspace tracking, IEEE Transactions on Signal Processing, 45 (1997), 2582-2586. Google Scholar |
[2] |
C. L. Fancourt and J. C. Principe,
Competitive principal component analysis for locally stationary time series, IEEE Transactions on Signal Processing, 46 (1998), 3068-3081.
doi: 10.1109/78.726819. |
[3] |
J. B. O. S. Filho and P. S. R. Diniz,
A fixed-point online kernel principal component extraction algorithm, IEEE Transactions on Signal Processing, 65 (2017), 6244-6259.
doi: 10.1109/TSP.2017.2750119. |
[4] |
I. A. Guimarães and A. C. Neto,
Estimation in polytomous logistic model: Comparison of methods, Journal of Industrial and Management Optimization, Journal of Industrial and Management Optimization, 5 (2009), 239-252.
doi: 10.3934/jimo.2009.5.239. |
[5] |
R. He, B. G. Hu, W. S. Zheng and X. W. Kong,
Robust principal component analysis based on maximum correntropy criterion, IEEE Transactions on Image Processing, 20 (2011), 1485-1494.
doi: 10.1109/TIP.2010.2103949. |
[6] |
S. M. Huang and J. F. Yang,
Improved principal component regression for face recognition under illumination variations, IEEE Signal Processing Letters, 19 (2012), 179-182.
doi: 10.1109/LSP.2012.2185492. |
[7] |
J. Kang, X. Lin and G. Yang, Research of multi-scale PCA algorithm for face recognition, International Conference on Information and Communications Technologies, ICT, (2015), 1–5. Google Scholar |
[8] |
M. S. Kang, J. H. Bae, B. S. Kang and K. T. Kim,
ISAR cross-range scaling using iterative processing via principal component analysis and bisection algorithm, IEEE Transactions on Signal Processing, 64 (2016), 3909-3918.
doi: 10.1109/TSP.2016.2552511. |
[9] |
S. Karamizadeh, S. M. Abdullah, A. A. Manaf and M. Zamani,
An overview of principal component analysis, Journal of Signal and Information Processing, 4 (2013), 173-175.
doi: 10.4236/jsip.2013.43B031. |
[10] |
H. C. Liu, S. T. Li and L. Y. Fang, Robust object tracking based on principal component analysis and local sparse representation, IEEE Transactions on Instrumentation and Measurement, 64 (2015), 2863-2875. Google Scholar |
[11] |
Z. Li, Q. L. Ye, Y. T. Guo, Z. K. Tian, B. W. K. Ling and R. W. K. Lam, Wearable non-invasive blood glucose estimation via empirical mode decomposition based hierarchical multiresolution analysis and random forest, 23rd IEEE International Conference on Digital Signal Processing, DSP, (2018), 1–5.
doi: 10.1109/ICDSP.2018.8631545. |
[12] |
A. Moradi, J. Razmi, R. Babazadeh and A. Sabbaghnia,
An integrated Principal Component Analysis and multi-objective mathematical programming approach to agile supply chain network design under uncertainty, Journal of Industrial and Management Optimization, 15 (2019), 855-879.
doi: 10.3934/jimo.2018074. |
[13] |
E. M. Moreno, Non-invasive estimate of blood glucose and blood pressure from a photoplethysmography by means of machine learning techniques, Artificial Intelligence in Medicine, 53 (2011), 127-138. Google Scholar |
[14] |
P. P. Markopulos, G. N. Karystinos and D. A. Pados,
Optimal algorithms for L1-subpace signal processing, IEEE Transactions on Signal Processing, 62 (2014), 5046-5058.
doi: 10.1109/TSP.2014.2338077. |
[15] |
O. Ozgonenel and T. Yalcin, Principal component analysis (PCA) based neural network for motor protection, 10th IET International Conference on Developments in Power System Protection, DPSP, (2010), 1–5.
doi: 10.1049/cp.2010.0252. |
[16] |
S. Ouyang and Z. Bao,
Fast principal component extraction by a weighted information criterion, IEEE Transactions on Signal Processing, 50 (2002), 1994-2002.
doi: 10.1109/TSP.2002.800395. |
[17] |
Y. M. Ren, L. Liao and S. J. Maybank,
Hyperspectral image spectral-spatial feature extraction via tensor principal component analysis, IEEE Geoscience and Remote Sensing Letters, 14 (2017), 1431-1435.
doi: 10.1109/LGRS.2017.2686878. |
[18] |
C. E. Thomaz and G. A. Giraldi,
A new ranking method for principal components analysis and its application to face image analysis, Image and Vision Computing, 28 (2009), 902-913.
doi: 10.1016/j.imavis.2009.11.005. |
[19] |
G. Tang and A. Nehorai,
Constrained Cramér-Rao bound on robust principal component analysis, IEEE Transactions on Signal Processing, 59 (2011), 5070-5076.
doi: 10.1109/TSP.2011.2161984. |
[20] |
X. C Xiu, Y. Yang, W.Q. Liu, L.C. Kong and M.J. Shang,
An improved total variation regularized RPCA for moving object detection with dynamic background, Journal of Industrial and Management Optimization, 13 (2017), 1-14.
doi: 10.3934/jimo.2019024. |
[21] |
S. Y. Yi, Z. H. Lai, Z. Y. He, Y. Cheung and Y. Liu,
Joint sparse principal component analysis, Pattern Recognition, 61 (2016), 524-536.
doi: 10.1016/j.patcog.2016.08.025. |
[22] |
A. Zare, A. Ozdemir, M. A. Iwen and S. Aviyente,
Extension of PCA to higher order data structures: An introduction to tensors, tensor decompositions, and tensor PCA, Proceedings of the IEEE, 106 (2018), 1341-1358.
doi: 10.1109/JPROC.2018.2848209. |
[23] |
H. Zou and L. Xue,
A selective overview of sparse principal component analysis, Proceedings of the IEEE, 106 (2018), 1311-1320.
doi: 10.1109/JPROC.2018.2846588. |

[1] |
Shenglan Xie, Maoan Han, Peng Zhu. A posteriori error estimate of weak Galerkin fem for second order elliptic problem with mixed boundary condition. Discrete & Continuous Dynamical Systems - B, 2020 doi: 10.3934/dcdsb.2020340 |
[2] |
Ningyu Sha, Lei Shi, Ming Yan. Fast algorithms for robust principal component analysis with an upper bound on the rank. Inverse Problems & Imaging, 2021, 15 (1) : 109-128. doi: 10.3934/ipi.2020067 |
[3] |
Juhua Shi, Feida Jiang. The degenerate Monge-Ampère equations with the Neumann condition. Communications on Pure & Applied Analysis, , () : -. doi: 10.3934/cpaa.2020297 |
[4] |
S. Sadeghi, H. Jafari, S. Nemati. Solving fractional Advection-diffusion equation using Genocchi operational matrix based on Atangana-Baleanu derivative. Discrete & Continuous Dynamical Systems - S, 2020 doi: 10.3934/dcdss.2020435 |
[5] |
Manil T. Mohan. First order necessary conditions of optimality for the two dimensional tidal dynamics system. Mathematical Control & Related Fields, 2020 doi: 10.3934/mcrf.2020045 |
[6] |
Yoshitsugu Kabeya. Eigenvalues of the Laplace-Beltrami operator under the homogeneous Neumann condition on a large zonal domain in the unit sphere. Discrete & Continuous Dynamical Systems - A, 2020, 40 (6) : 3529-3559. doi: 10.3934/dcds.2020040 |
[7] |
Larissa Fardigola, Kateryna Khalina. Controllability problems for the heat equation on a half-axis with a bounded control in the Neumann boundary condition. Mathematical Control & Related Fields, 2021, 11 (1) : 211-236. doi: 10.3934/mcrf.2020034 |
[8] |
Darko Dimitrov, Hosam Abdo. Tight independent set neighborhood union condition for fractional critical deleted graphs and ID deleted graphs. Discrete & Continuous Dynamical Systems - S, 2019, 12 (4&5) : 711-721. doi: 10.3934/dcdss.2019045 |
[9] |
Md. Masum Murshed, Kouta Futai, Masato Kimura, Hirofumi Notsu. Theoretical and numerical studies for energy estimates of the shallow water equations with a transmission boundary condition. Discrete & Continuous Dynamical Systems - S, 2021, 14 (3) : 1063-1078. doi: 10.3934/dcdss.2020230 |
[10] |
Tomáš Bodnár, Philippe Fraunié, Petr Knobloch, Hynek Řezníček. Numerical evaluation of artificial boundary condition for wall-bounded stably stratified flows. Discrete & Continuous Dynamical Systems - S, 2021, 14 (3) : 785-801. doi: 10.3934/dcdss.2020333 |
[11] |
Weisong Dong, Chang Li. Second order estimates for complex Hessian equations on Hermitian manifolds. Discrete & Continuous Dynamical Systems - A, 2020 doi: 10.3934/dcds.2020377 |
[12] |
Min Xi, Wenyu Sun, Jun Chen. Survey of derivative-free optimization. Numerical Algebra, Control & Optimization, 2020, 10 (4) : 537-555. doi: 10.3934/naco.2020050 |
[13] |
Yuyuan Ouyang, Trevor Squires. Some worst-case datasets of deterministic first-order methods for solving binary logistic regression. Inverse Problems & Imaging, 2021, 15 (1) : 63-77. doi: 10.3934/ipi.2020047 |
[14] |
Shasha Hu, Yihong Xu, Yuhan Zhang. Second-Order characterizations for set-valued equilibrium problems with variable ordering structures. Journal of Industrial & Management Optimization, 2020 doi: 10.3934/jimo.2020164 |
[15] |
Soniya Singh, Sumit Arora, Manil T. Mohan, Jaydev Dabas. Approximate controllability of second order impulsive systems with state-dependent delay in Banach spaces. Evolution Equations & Control Theory, 2020 doi: 10.3934/eect.2020103 |
[16] |
Liupeng Wang, Yunqing Huang. Error estimates for second-order SAV finite element method to phase field crystal model. Electronic Research Archive, 2021, 29 (1) : 1735-1752. doi: 10.3934/era.2020089 |
[17] |
Elimhan N. Mahmudov. Infimal convolution and duality in convex optimal control problems with second order evolution differential inclusions. Evolution Equations & Control Theory, 2021, 10 (1) : 37-59. doi: 10.3934/eect.2020051 |
[18] |
Ying Lv, Yan-Fang Xue, Chun-Lei Tang. Ground state homoclinic orbits for a class of asymptotically periodic second-order Hamiltonian systems. Discrete & Continuous Dynamical Systems - B, 2021, 26 (3) : 1627-1652. doi: 10.3934/dcdsb.2020176 |
[19] |
Mokhtar Bouloudene, Manar A. Alqudah, Fahd Jarad, Yassine Adjabi, Thabet Abdeljawad. Nonlinear singular $ p $ -Laplacian boundary value problems in the frame of conformable derivative. Discrete & Continuous Dynamical Systems - S, 2020 doi: 10.3934/dcdss.2020442 |
[20] |
Olivier Ley, Erwin Topp, Miguel Yangari. Some results for the large time behavior of Hamilton-Jacobi equations with Caputo time derivative. Discrete & Continuous Dynamical Systems - A, 2021 doi: 10.3934/dcds.2021007 |
2019 Impact Factor: 1.366
Tools
Metrics
Other articles
by authors
[Back to Top]