American Institute of Mathematical Sciences

January  2016, 12(1): 251-268. doi: 10.3934/jimo.2016.12.251

A novel discriminant minimum class locality preserving canonical correlation analysis and its applications

 1 Institute of Metrology and Computational Science, China Jiliang University, Hangzhou, 310018, Zhejiang Province, China, China, China

Received  October 2013 Revised  January 2015 Published  April 2015

Canonical correlation analysis(CCA) is a well-known technique for simultaneously reducing two relevant data sets, and finding maximal correlation between them. However, it fails to preserve the local structure of each data set, as well as the global discriminant ability, which are important in real applications. In this paper, a new CCA model, called discriminant minimum class locality preserving canonical correlation analysis(called as DMPCCA) is proposed. The proposed method introduces locall structure information and global discriminant information into the classical CCA and considers a optimal combination of intra-class locality preserving, global discriminant ability and the maximal correlation between two sets. The experiments on data visualization, web image retrieval and face recognition validate the effectiveness of the proposed method.
Citation: Yubo Yuan, Chenglong Ma, Dongmei Pu. A novel discriminant minimum class locality preserving canonical correlation analysis and its applications. Journal of Industrial & Management Optimization, 2016, 12 (1) : 251-268. doi: 10.3934/jimo.2016.12.251
References:
 [1] B. Abraham, et al., Dimensionality reduction approach to multivariate prediction,, Comput Stat Data Anal, 48 (2005), 5.  doi: 10.1016/j.csda.2003.11.021.  Google Scholar [2] N. E. Ayat, et al., Automatic model selection for the optimization of SVM kernels,, Pattern Recognition, 38 (2005), 1733.  doi: 10.1016/j.patcog.2005.03.011.  Google Scholar [3] P. N. Belhumeur, et al., Eigenfaces vs. fisherfaces: Recognition using class specific linear projection,, IEEE Transactions on Pattern Analysis and Machine Intelligence, 19 (1997), 711.  doi: 10.1109/34.598228.  Google Scholar [4] Q. N. Chen, et al., Hierarchical multi-view fisher discriminant analysis,, International Conference on Neural Information Processing, 5864 (2009), 289.  doi: 10.1007/978-3-642-10684-2_32.  Google Scholar [5] T. Diethe, et al., Constructing nonlinear discriminants from multiple data views,, Machine Learning and Knowledge Discovery in Databases, 6321 (2010), 328.  doi: 10.1007/978-3-642-15880-3_27.  Google Scholar [6] J. H. Friedman, et al., Regularized discriminant analysis,, Journal of the American Statistics Association, 84 (1989), 165.  doi: 10.1080/01621459.1989.10478752.  Google Scholar [7] D. R. Hardoon, Sparse canonical correlation analysis,, Machine Learning, 83 (2011), 331.  doi: 10.1007/s10994-010-5222-7.  Google Scholar [8] D. R. Hardoon, et al., Canonical correlation analysis: An overview with application to learning methods,, Neural Computation, 16 (2004), 2639.  doi: 10.1162/0899766042321814.  Google Scholar [9] X. He, et al., Local Preserving Projections,, Advances in Neural Information Processing Systems, (2003).   Google Scholar [10] X. He, et al., Face recognition using Laplicianfaces,, IEEE Transactions on Pattern Analysis and Machine Intelligence, 27 (2005), 328.   Google Scholar [11] L. Hoegaerts, et al., Subset based least squares subspace regression in RKHS,, Neurocomputing, 63 (2005), 293.  doi: 10.1016/j.neucom.2004.04.013.  Google Scholar [12] Y. J. Huang, et al., Protein NMR recall, precision, and F-measure scores (RPF scores): Structure quality assessment measures based on information retrieval statistics,, Journal of the American Chemical Society, 127 (2005), 1665.   Google Scholar [13] H. Hotelling, Relations between two sets of variates,, Biometrika, 28 (1936), 312.  doi: 10.2307/2333955.  Google Scholar [14] Z. Ji, et al., Rank canonical correlation analysis and its application in visual search reranking,, Signal Processing, 93 (2013), 2352.  doi: 10.1016/j.sigpro.2012.05.006.  Google Scholar [15] X. Y. Jing, et al., Color image canonical correlation analysis for face feature extraction and recognition,, Signal Processing, 91 (2011), 2132.  doi: 10.1016/j.sigpro.2011.02.016.  Google Scholar [16] E. Kitani, et al., Um Tutorial sobre Analise de Componentes Principais para o Reconhecimento Automatico de Faces [R/OL],, , (2006).   Google Scholar [17] P. L. Lai, et al., Kernel and nonlinear canonical correlation analysis,, International Journal of Neural Systems, 10 (2000), 365.  doi: 10.1016/S0129-0657(00)00034-X.  Google Scholar [18] Y. Liu, et al., A survey of content-based image retrieval with high-level semantics,, Pattern Recognition, 40 (2007), 262.  doi: 10.1016/j.patcog.2006.04.045.  Google Scholar [19] C. D. Manning, et al., Introduction to Information Retrieval,, Cambridge: Cambridge university press, (2008).  doi: 10.1017/CBO9780511809071.  Google Scholar [20] T. Melzer, et al., Appearance models based on kernel canonical correlation analysis,, Pattern recognition, 36 (2003), 1961.  doi: 10.1016/S0031-3203(03)00058-X.  Google Scholar [21] T. Melzer, et al., Appearance models based on kernel canonical correlation analysis,, Pattern Recognition, 36 (2003), 1961.  doi: 10.1016/S0031-3203(03)00058-X.  Google Scholar [22] A. A. Nielsen, et al., Multiset canonical correlations analysis and multispectral truly multitemporal remote sensing data,, IEEE Transactions on Image Processing, 11 (2002), 293.  doi: 10.1109/83.988962.  Google Scholar [23] S. Roweis, et al., Nonlinear dimensionality reduction by local linear embedding,, Science, 290 (2000), 2323.  doi: 10.1126/science.290.5500.2323.  Google Scholar [24] M. Ortega, et al., Supporting ranked boolean similarity queries in MARS,, IEEE Transaction on Knowledge and Data Engineering, 10 (1998), 905.  doi: 10.1109/69.738357.  Google Scholar [25] N. Otopal, et al., Restricted kernel canonical correlation analysis,, Linear Algebra and its Applications, 437 (2012), 1.  doi: 10.1016/j.laa.2012.02.014.  Google Scholar [26] O. A. B. Penatti, et al., Comparative study of global color and texture descriptors for web image retrieval,, Journal of Visual Communication and Image Representation, 23 (2012), 359.  doi: 10.1016/j.jvcir.2011.11.002.  Google Scholar [27] Y. Peng, et al., Semi-supervised kernel canonical correlation analysis,, Journal of Software, 19 (2008), 2822.   Google Scholar [28] Y. Peng, et al., A new canonical correlation analysis algorithm with local discrimination,, Neural Processing Letters, 31 (2010), 1.  doi: 10.1007/s11063-009-9123-3.  Google Scholar [29] R. Pless, et al., A Survey of Manifold Learning,, PIPSJ Transactions on Computer Vision and Applications, 1 (2009), 83.   Google Scholar [30] F. S. Samaria, et al., Parameterisation of a stochastic model for human face identification,, In Second IEEE Workshop on Applications of Computer Vision, (1994), 138.  doi: 10.1109/ACV.1994.341300.  Google Scholar [31] A. Sharma, et al., Generalized Multiview Analysis: A discriminative latent space,, IEEE Conference on Computer Vision and Pattern Recognition, (2012), 2160.  doi: 10.1109/CVPR.2012.6247923.  Google Scholar [32] L. Sun, et al., Canonical correlation analysis for multilabel classification: A least-squares formulation, extensions, and analysis,, IEEE Transactions on Pattern Analysis and Machine Intelligence, 33 (2011), 194.   Google Scholar [33] Q. Sun, et al., A new method of feature fusion and its application in image recognition,, Pattern Recognition, 38 (2005), 2437.  doi: 10.1016/j.patcog.2004.12.013.  Google Scholar [34] T. K. Sun, et al., A novel method of combined feature extraction for recognition,, IEEE Conference on Data Mining, (2008), 1043.  doi: 10.1109/ICDM.2008.28.  Google Scholar [35] T. K. Sun, et al., Locality preserving CCA with applications to data visualization and pose estimation,, Image and Vision Computing, 25 (2007), 531.  doi: 10.1016/j.imavis.2006.04.014.  Google Scholar [36] M. Turk, et al., Eigenfaces for recognition,, Journal of Cognitive Neuroscience, 3 (1991), 71.  doi: 10.1162/jocn.1991.3.1.71.  Google Scholar [37] N. Vlassis, et al., Supervised linear feature extraction for mobile robot localization,, Proceedings of the IEEE international conference on robotics and automation, 3 (2000), 2979.  doi: 10.1109/ROBOT.2000.846480.  Google Scholar [38] Y. H. Yan, et al., A novel multiset integrated canonical correlation analysis framework and its application in feature fusion,, Pattern Recognition, 44 (2011), 1031.  doi: 10.1016/j.patcog.2010.11.004.  Google Scholar [39] X. Zhu, et al., Dimensionality reduction by mixed kernel canonical correlation analysis,, Pattern Recognition, 45 (2012), 3003.  doi: 10.1016/j.patcog.2012.02.007.  Google Scholar [40] J. Yang, et al., Feature fusion: Parallel strategy vs. serial strategy,, Pattern Recognition, 36 (2003), 1369.  doi: 10.1016/S0031-3203(02)00262-5.  Google Scholar [41] W. W. Yu, et al., Face recognition using discriminant locality preserving projections,, Image and Vision computing, 24 (2006), 239.  doi: 10.1016/j.imavis.2005.11.006.  Google Scholar [42] Y. B. Yuan, Canonical duality solution for alternating support vector machine,, Journal of Industrial and Management Optimization, 8 (2012), 611.  doi: 10.3934/jimo.2012.8.611.  Google Scholar [43] X. Zhang, et al., Discriminative locality preserving canonical correlation analysis,, Pattern Recognition, 321 (2012), 341.  doi: 10.1007/978-3-642-33506-8_43.  Google Scholar [44] UCI, UCI Repository of machine learning databases,, , ().   Google Scholar

show all references

References:
 [1] B. Abraham, et al., Dimensionality reduction approach to multivariate prediction,, Comput Stat Data Anal, 48 (2005), 5.  doi: 10.1016/j.csda.2003.11.021.  Google Scholar [2] N. E. Ayat, et al., Automatic model selection for the optimization of SVM kernels,, Pattern Recognition, 38 (2005), 1733.  doi: 10.1016/j.patcog.2005.03.011.  Google Scholar [3] P. N. Belhumeur, et al., Eigenfaces vs. fisherfaces: Recognition using class specific linear projection,, IEEE Transactions on Pattern Analysis and Machine Intelligence, 19 (1997), 711.  doi: 10.1109/34.598228.  Google Scholar [4] Q. N. Chen, et al., Hierarchical multi-view fisher discriminant analysis,, International Conference on Neural Information Processing, 5864 (2009), 289.  doi: 10.1007/978-3-642-10684-2_32.  Google Scholar [5] T. Diethe, et al., Constructing nonlinear discriminants from multiple data views,, Machine Learning and Knowledge Discovery in Databases, 6321 (2010), 328.  doi: 10.1007/978-3-642-15880-3_27.  Google Scholar [6] J. H. Friedman, et al., Regularized discriminant analysis,, Journal of the American Statistics Association, 84 (1989), 165.  doi: 10.1080/01621459.1989.10478752.  Google Scholar [7] D. R. Hardoon, Sparse canonical correlation analysis,, Machine Learning, 83 (2011), 331.  doi: 10.1007/s10994-010-5222-7.  Google Scholar [8] D. R. Hardoon, et al., Canonical correlation analysis: An overview with application to learning methods,, Neural Computation, 16 (2004), 2639.  doi: 10.1162/0899766042321814.  Google Scholar [9] X. He, et al., Local Preserving Projections,, Advances in Neural Information Processing Systems, (2003).   Google Scholar [10] X. He, et al., Face recognition using Laplicianfaces,, IEEE Transactions on Pattern Analysis and Machine Intelligence, 27 (2005), 328.   Google Scholar [11] L. Hoegaerts, et al., Subset based least squares subspace regression in RKHS,, Neurocomputing, 63 (2005), 293.  doi: 10.1016/j.neucom.2004.04.013.  Google Scholar [12] Y. J. Huang, et al., Protein NMR recall, precision, and F-measure scores (RPF scores): Structure quality assessment measures based on information retrieval statistics,, Journal of the American Chemical Society, 127 (2005), 1665.   Google Scholar [13] H. Hotelling, Relations between two sets of variates,, Biometrika, 28 (1936), 312.  doi: 10.2307/2333955.  Google Scholar [14] Z. Ji, et al., Rank canonical correlation analysis and its application in visual search reranking,, Signal Processing, 93 (2013), 2352.  doi: 10.1016/j.sigpro.2012.05.006.  Google Scholar [15] X. Y. Jing, et al., Color image canonical correlation analysis for face feature extraction and recognition,, Signal Processing, 91 (2011), 2132.  doi: 10.1016/j.sigpro.2011.02.016.  Google Scholar [16] E. Kitani, et al., Um Tutorial sobre Analise de Componentes Principais para o Reconhecimento Automatico de Faces [R/OL],, , (2006).   Google Scholar [17] P. L. Lai, et al., Kernel and nonlinear canonical correlation analysis,, International Journal of Neural Systems, 10 (2000), 365.  doi: 10.1016/S0129-0657(00)00034-X.  Google Scholar [18] Y. Liu, et al., A survey of content-based image retrieval with high-level semantics,, Pattern Recognition, 40 (2007), 262.  doi: 10.1016/j.patcog.2006.04.045.  Google Scholar [19] C. D. Manning, et al., Introduction to Information Retrieval,, Cambridge: Cambridge university press, (2008).  doi: 10.1017/CBO9780511809071.  Google Scholar [20] T. Melzer, et al., Appearance models based on kernel canonical correlation analysis,, Pattern recognition, 36 (2003), 1961.  doi: 10.1016/S0031-3203(03)00058-X.  Google Scholar [21] T. Melzer, et al., Appearance models based on kernel canonical correlation analysis,, Pattern Recognition, 36 (2003), 1961.  doi: 10.1016/S0031-3203(03)00058-X.  Google Scholar [22] A. A. Nielsen, et al., Multiset canonical correlations analysis and multispectral truly multitemporal remote sensing data,, IEEE Transactions on Image Processing, 11 (2002), 293.  doi: 10.1109/83.988962.  Google Scholar [23] S. Roweis, et al., Nonlinear dimensionality reduction by local linear embedding,, Science, 290 (2000), 2323.  doi: 10.1126/science.290.5500.2323.  Google Scholar [24] M. Ortega, et al., Supporting ranked boolean similarity queries in MARS,, IEEE Transaction on Knowledge and Data Engineering, 10 (1998), 905.  doi: 10.1109/69.738357.  Google Scholar [25] N. Otopal, et al., Restricted kernel canonical correlation analysis,, Linear Algebra and its Applications, 437 (2012), 1.  doi: 10.1016/j.laa.2012.02.014.  Google Scholar [26] O. A. B. Penatti, et al., Comparative study of global color and texture descriptors for web image retrieval,, Journal of Visual Communication and Image Representation, 23 (2012), 359.  doi: 10.1016/j.jvcir.2011.11.002.  Google Scholar [27] Y. Peng, et al., Semi-supervised kernel canonical correlation analysis,, Journal of Software, 19 (2008), 2822.   Google Scholar [28] Y. Peng, et al., A new canonical correlation analysis algorithm with local discrimination,, Neural Processing Letters, 31 (2010), 1.  doi: 10.1007/s11063-009-9123-3.  Google Scholar [29] R. Pless, et al., A Survey of Manifold Learning,, PIPSJ Transactions on Computer Vision and Applications, 1 (2009), 83.   Google Scholar [30] F. S. Samaria, et al., Parameterisation of a stochastic model for human face identification,, In Second IEEE Workshop on Applications of Computer Vision, (1994), 138.  doi: 10.1109/ACV.1994.341300.  Google Scholar [31] A. Sharma, et al., Generalized Multiview Analysis: A discriminative latent space,, IEEE Conference on Computer Vision and Pattern Recognition, (2012), 2160.  doi: 10.1109/CVPR.2012.6247923.  Google Scholar [32] L. Sun, et al., Canonical correlation analysis for multilabel classification: A least-squares formulation, extensions, and analysis,, IEEE Transactions on Pattern Analysis and Machine Intelligence, 33 (2011), 194.   Google Scholar [33] Q. Sun, et al., A new method of feature fusion and its application in image recognition,, Pattern Recognition, 38 (2005), 2437.  doi: 10.1016/j.patcog.2004.12.013.  Google Scholar [34] T. K. Sun, et al., A novel method of combined feature extraction for recognition,, IEEE Conference on Data Mining, (2008), 1043.  doi: 10.1109/ICDM.2008.28.  Google Scholar [35] T. K. Sun, et al., Locality preserving CCA with applications to data visualization and pose estimation,, Image and Vision Computing, 25 (2007), 531.  doi: 10.1016/j.imavis.2006.04.014.  Google Scholar [36] M. Turk, et al., Eigenfaces for recognition,, Journal of Cognitive Neuroscience, 3 (1991), 71.  doi: 10.1162/jocn.1991.3.1.71.  Google Scholar [37] N. Vlassis, et al., Supervised linear feature extraction for mobile robot localization,, Proceedings of the IEEE international conference on robotics and automation, 3 (2000), 2979.  doi: 10.1109/ROBOT.2000.846480.  Google Scholar [38] Y. H. Yan, et al., A novel multiset integrated canonical correlation analysis framework and its application in feature fusion,, Pattern Recognition, 44 (2011), 1031.  doi: 10.1016/j.patcog.2010.11.004.  Google Scholar [39] X. Zhu, et al., Dimensionality reduction by mixed kernel canonical correlation analysis,, Pattern Recognition, 45 (2012), 3003.  doi: 10.1016/j.patcog.2012.02.007.  Google Scholar [40] J. Yang, et al., Feature fusion: Parallel strategy vs. serial strategy,, Pattern Recognition, 36 (2003), 1369.  doi: 10.1016/S0031-3203(02)00262-5.  Google Scholar [41] W. W. Yu, et al., Face recognition using discriminant locality preserving projections,, Image and Vision computing, 24 (2006), 239.  doi: 10.1016/j.imavis.2005.11.006.  Google Scholar [42] Y. B. Yuan, Canonical duality solution for alternating support vector machine,, Journal of Industrial and Management Optimization, 8 (2012), 611.  doi: 10.3934/jimo.2012.8.611.  Google Scholar [43] X. Zhang, et al., Discriminative locality preserving canonical correlation analysis,, Pattern Recognition, 321 (2012), 341.  doi: 10.1007/978-3-642-33506-8_43.  Google Scholar [44] UCI, UCI Repository of machine learning databases,, , ().   Google Scholar
 [1] Jia Cai, Junyi Huo. Sparse generalized canonical correlation analysis via linearized Bregman method. Communications on Pure & Applied Analysis, 2020, 19 (8) : 3933-3945. doi: 10.3934/cpaa.2020173 [2] Sohana Jahan. Discriminant analysis of regularized multidimensional scaling. Numerical Algebra, Control & Optimization, 2020  doi: 10.3934/naco.2020024 [3] Wenjuan Jia, Yingjie Deng, Chenyang Xin, Xiaodong Liu, Witold Pedrycz. A classification algorithm with Linear Discriminant Analysis and Axiomatic Fuzzy Sets. Mathematical Foundations of Computing, 2019, 2 (1) : 73-81. doi: 10.3934/mfc.2019006 [4] Jian Luo, Shu-Cherng Fang, Yanqin Bai, Zhibin Deng. Fuzzy quadratic surface support vector machine based on fisher discriminant analysis. Journal of Industrial & Management Optimization, 2016, 12 (1) : 357-373. doi: 10.3934/jimo.2016.12.357 [5] Boling Guo, Guoli Zhou. Finite dimensionality of global attractor for the solutions to 3D viscous primitive equations of large-scale moist atmosphere. Discrete & Continuous Dynamical Systems - B, 2018, 23 (10) : 4305-4327. doi: 10.3934/dcdsb.2018160 [6] Fanwen Meng, Kiok Liang Teow, Kelvin Wee Sheng Teo, Chee Kheong Ooi, Seow Yian Tay. Predicting 72-hour reattendance in emergency departments using discriminant analysis via mixed integer programming with electronic medical records. Journal of Industrial & Management Optimization, 2019, 15 (2) : 947-962. doi: 10.3934/jimo.2018079 [7] Rentsen Enkhbat, Evgeniya A. Finkelstein, Anton S. Anikin, Alexandr Yu. Gornov. Global optimization reduction of generalized Malfatti's problem. Numerical Algebra, Control & Optimization, 2017, 7 (2) : 211-221. doi: 10.3934/naco.2017015 [8] Adrian Viorel, Cristian D. Alecsa, Titus O. Pinţa. Asymptotic analysis of a structure-preserving integrator for damped Hamiltonian systems. Discrete & Continuous Dynamical Systems - A, 2020  doi: 10.3934/dcds.2020407 [9] Claude Carlet, Khoongming Khoo, Chu-Wee Lim, Chuan-Wen Loe. On an improved correlation analysis of stream ciphers using multi-output Boolean functions and the related generalized notion of nonlinearity. Advances in Mathematics of Communications, 2008, 2 (2) : 201-221. doi: 10.3934/amc.2008.2.201 [10] Lassi Roininen, Markku S. Lehtinen, Sari Lasanen, Mikko Orispää, Markku Markkanen. Correlation priors. Inverse Problems & Imaging, 2011, 5 (1) : 167-184. doi: 10.3934/ipi.2011.5.167 [11] Hayato Chiba, Georgi S. Medvedev. The mean field analysis of the kuramoto model on graphs Ⅱ. asymptotic stability of the incoherent state, center manifold reduction, and bifurcations. Discrete & Continuous Dynamical Systems - A, 2019, 39 (7) : 3897-3921. doi: 10.3934/dcds.2019157 [12] Marc Massot. Singular perturbation analysis for the reduction of complex chemistry in gaseous mixtures using the entropic structure. Discrete & Continuous Dynamical Systems - B, 2002, 2 (3) : 433-456. doi: 10.3934/dcdsb.2002.2.433 [13] Georg Vossen, Stefan Volkwein. Model reduction techniques with a-posteriori error analysis for linear-quadratic optimal control problems. Numerical Algebra, Control & Optimization, 2012, 2 (3) : 465-485. doi: 10.3934/naco.2012.2.465 [14] Alexander Sakhnovich. Dynamical canonical systems and their explicit solutions. Discrete & Continuous Dynamical Systems - A, 2017, 37 (3) : 1679-1689. doi: 10.3934/dcds.2017069 [15] Nikita Selinger. Topological characterization of canonical Thurston obstructions. Journal of Modern Dynamics, 2013, 7 (1) : 99-117. doi: 10.3934/jmd.2013.7.99 [16] Peter Scott and Gadde A. Swarup. Regular neighbourhoods and canonical decompositions for groups. Electronic Research Announcements, 2002, 8: 20-28. [17] Marilena N. Poulou, Nikolaos M. Stavrakakis. Finite dimensionality of a Klein-Gordon-Schrödinger type system. Discrete & Continuous Dynamical Systems - S, 2009, 2 (1) : 149-161. doi: 10.3934/dcdss.2009.2.149 [18] Tao Feng, Zhipeng Qiu. Global analysis of a stochastic TB model with vaccination and treatment. Discrete & Continuous Dynamical Systems - B, 2019, 24 (6) : 2923-2939. doi: 10.3934/dcdsb.2018292 [19] Mohamed Dellal, Bachir Bar. Global analysis of a model of competition in the chemostat with internal inhibitor. Discrete & Continuous Dynamical Systems - B, 2021, 26 (2) : 1129-1148. doi: 10.3934/dcdsb.2020156 [20] Frederic Heihoff. Global mass-preserving solutions for a two-dimensional chemotaxis system with rotational flux components coupled with a full Navier–Stokes equation. Discrete & Continuous Dynamical Systems - B, 2020, 25 (12) : 4703-4719. doi: 10.3934/dcdsb.2020120

2019 Impact Factor: 1.366

Metrics

• PDF downloads (88)
• HTML views (0)
• Cited by (1)

Other articlesby authors

• on AIMS
• on Google Scholar

[Back to Top]