A novel discriminant minimum class locality preserving canonical correlation analysis and its applications

Previous Article
Quadratic optimization over a polyhedral cone
JIMO Home
This Issue
Next Article
Time consistent policy of multi-period mean-variance problem in stochastic markets

January 2016, 12(1): 251-268. doi: 10.3934/jimo.2016.12.251

A novel discriminant minimum class locality preserving canonical correlation analysis and its applications

Yubo Yuan ^1, , Chenglong Ma ^1, and Dongmei Pu ^1,

1.	Institute of Metrology and Computational Science, China Jiliang University, Hangzhou, 310018, Zhejiang Province, China, China, China

Received October 2013 Revised January 2015 Published April 2015

Abstract
Related Papers

Canonical correlation analysis(CCA) is a well-known technique for simultaneously reducing two relevant data sets, and finding maximal correlation between them. However, it fails to preserve the local structure of each data set, as well as the global discriminant ability, which are important in real applications. In this paper, a new CCA model, called discriminant minimum class locality preserving canonical correlation analysis(called as DMPCCA) is proposed. The proposed method introduces locall structure information and global discriminant information into the classical CCA and considers a optimal combination of intra-class locality preserving, global discriminant ability and the maximal correlation between two sets. The experiments on data visualization, web image retrieval and face recognition validate the effectiveness of the proposed method.

Keywords: Canonical correlation analysis, global discriminant, locality preserving, dimensionality reduction..

Mathematics Subject Classification: Primary: 68T10; Secondary: 90C2.

Citation: Yubo Yuan, Chenglong Ma, Dongmei Pu. A novel discriminant minimum class locality preserving canonical correlation analysis and its applications. Journal of Industrial and Management Optimization, 2016, 12 (1) : 251-268. doi: 10.3934/jimo.2016.12.251

References:

[1]	B. Abraham, et al., Dimensionality reduction approach to multivariate prediction, Comput Stat Data Anal, 48 (2005), 5-16. doi: 10.1016/j.csda.2003.11.021.
[2]	N. E. Ayat, et al., Automatic model selection for the optimization of SVM kernels, Pattern Recognition, 38 (2005), 1733-1745. doi: 10.1016/j.patcog.2005.03.011.
[3]	P. N. Belhumeur, et al., Eigenfaces vs. fisherfaces: Recognition using class specific linear projection, IEEE Transactions on Pattern Analysis and Machine Intelligence, 19 (1997), 711-720. doi: 10.1109/34.598228.
[4]	Q. N. Chen, et al., Hierarchical multi-view fisher discriminant analysis, International Conference on Neural Information Processing, 5864 (2009), 289-298. doi: 10.1007/978-3-642-10684-2_32.
[5]	T. Diethe, et al., Constructing nonlinear discriminants from multiple data views, Machine Learning and Knowledge Discovery in Databases, 6321 (2010), 328-343. doi: 10.1007/978-3-642-15880-3_27.
[6]	J. H. Friedman, et al., Regularized discriminant analysis, Journal of the American Statistics Association, 84 (1989), 165-175. doi: 10.1080/01621459.1989.10478752.
[7]	D. R. Hardoon, Sparse canonical correlation analysis, Machine Learning, 83 (2011), 331-353. doi: 10.1007/s10994-010-5222-7.
[8]	D. R. Hardoon, et al., Canonical correlation analysis: An overview with application to learning methods, Neural Computation, 16 (2004), 2639-2664. doi: 10.1162/0899766042321814.
[9]	X. He, et al., Local Preserving Projections, Advances in Neural Information Processing Systems, MIT Press, 2003.
[10]	X. He, et al., Face recognition using Laplicianfaces, IEEE Transactions on Pattern Analysis and Machine Intelligence, 27 (2005), 328-340.
[11]	L. Hoegaerts, et al., Subset based least squares subspace regression in RKHS, Neurocomputing, 63 (2005), 293-323. doi: 10.1016/j.neucom.2004.04.013.
[12]	Y. J. Huang, et al., Protein NMR recall, precision, and F-measure scores (RPF scores): Structure quality assessment measures based on information retrieval statistics, Journal of the American Chemical Society, 127 (2005), 1665-1674.
[13]	H. Hotelling, Relations between two sets of variates, Biometrika, 28 (1936), 312-377. doi: 10.2307/2333955.
[14]	Z. Ji, et al., Rank canonical correlation analysis and its application in visual search reranking, Signal Processing, 93 (2013), 2352-2360. doi: 10.1016/j.sigpro.2012.05.006.
[15]	X. Y. Jing, et al., Color image canonical correlation analysis for face feature extraction and recognition, Signal Processing, 91 (2011), 2132-2140. doi: 10.1016/j.sigpro.2011.02.016.
[16]	E. Kitani, et al., Um Tutorial sobre Analise de Componentes Principais para o Reconhecimento Automatico de Faces [R/OL], http://www.fei.edu.br/~cet, 2006.
[17]	P. L. Lai, et al., Kernel and nonlinear canonical correlation analysis, International Journal of Neural Systems, 10 (2000), 365-377. doi: 10.1016/S0129-0657(00)00034-X.
[18]	Y. Liu, et al., A survey of content-based image retrieval with high-level semantics, Pattern Recognition, 40 (2007), 262-282. doi: 10.1016/j.patcog.2006.04.045.
[19]	C. D. Manning, et al., Introduction to Information Retrieval, Cambridge: Cambridge university press, 2008. doi: 10.1017/CBO9780511809071.
[20]	T. Melzer, et al., Appearance models based on kernel canonical correlation analysis, Pattern recognition, 36 (2003), 1961-1971. doi: 10.1016/S0031-3203(03)00058-X.
[21]	T. Melzer, et al., Appearance models based on kernel canonical correlation analysis, Pattern Recognition, 36 (2003), 1961-1971. doi: 10.1016/S0031-3203(03)00058-X.
[22]	A. A. Nielsen, et al., Multiset canonical correlations analysis and multispectral truly multitemporal remote sensing data, IEEE Transactions on Image Processing, 11 (2002), 293-305. doi: 10.1109/83.988962.
[23]	S. Roweis, et al., Nonlinear dimensionality reduction by local linear embedding, Science, 290 (2000), 2323-2326. doi: 10.1126/science.290.5500.2323.
[24]	M. Ortega, et al., Supporting ranked boolean similarity queries in MARS, IEEE Transaction on Knowledge and Data Engineering, 10 (1998), 905-925. doi: 10.1109/69.738357.
[25]	N. Otopal, et al., Restricted kernel canonical correlation analysis, Linear Algebra and its Applications, 437 (2012), 1-13. doi: 10.1016/j.laa.2012.02.014.
[26]	O. A. B. Penatti, et al., Comparative study of global color and texture descriptors for web image retrieval, Journal of Visual Communication and Image Representation, 23 (2012), 359-380. doi: 10.1016/j.jvcir.2011.11.002.
[27]	Y. Peng, et al., Semi-supervised kernel canonical correlation analysis, Journal of Software, 19 (2008), 2822-2832.
[28]	Y. Peng, et al., A new canonical correlation analysis algorithm with local discrimination, Neural Processing Letters, 31 (2010), 1-15. doi: 10.1007/s11063-009-9123-3.
[29]	R. Pless, et al., A Survey of Manifold Learning, PIPSJ Transactions on Computer Vision and Applications, 1 (2009), 83-94.
[30]	F. S. Samaria, et al., Parameterisation of a stochastic model for human face identification, In Second IEEE Workshop on Applications of Computer Vision, (1994), 138-142. doi: 10.1109/ACV.1994.341300.
[31]	A. Sharma, et al., Generalized Multiview Analysis: A discriminative latent space, IEEE Conference on Computer Vision and Pattern Recognition, (2012), 2160-2167. doi: 10.1109/CVPR.2012.6247923.
[32]	L. Sun, et al., Canonical correlation analysis for multilabel classification: A least-squares formulation, extensions, and analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence, 33 (2011), 194-200.
[33]	Q. Sun, et al., A new method of feature fusion and its application in image recognition, Pattern Recognition, 38 (2005), 2437-2448. doi: 10.1016/j.patcog.2004.12.013.
[34]	T. K. Sun, et al., A novel method of combined feature extraction for recognition, IEEE Conference on Data Mining, (2008), 1043-1048. doi: 10.1109/ICDM.2008.28.
[35]	T. K. Sun, et al., Locality preserving CCA with applications to data visualization and pose estimation, Image and Vision Computing, 25 (2007), 531-543. doi: 10.1016/j.imavis.2006.04.014.
[36]	M. Turk, et al., Eigenfaces for recognition, Journal of Cognitive Neuroscience, 3 (1991), 71-86. doi: 10.1162/jocn.1991.3.1.71.
[37]	N. Vlassis, et al., Supervised linear feature extraction for mobile robot localization, Proceedings of the IEEE international conference on robotics and automation, 3 (2000), 2979-2984. doi: 10.1109/ROBOT.2000.846480.
[38]	Y. H. Yan, et al., A novel multiset integrated canonical correlation analysis framework and its application in feature fusion, Pattern Recognition, 44 (2011), 1031-1040. doi: 10.1016/j.patcog.2010.11.004.
[39]	X. Zhu, et al., Dimensionality reduction by mixed kernel canonical correlation analysis, Pattern Recognition, 45 (2012), 3003-3016. doi: 10.1016/j.patcog.2012.02.007.
[40]	J. Yang, et al., Feature fusion: Parallel strategy vs. serial strategy, Pattern Recognition, 36 (2003), 1369-1381. doi: 10.1016/S0031-3203(02)00262-5.
[41]	W. W. Yu, et al., Face recognition using discriminant locality preserving projections, Image and Vision computing, 24 (2006), 239-248. doi: 10.1016/j.imavis.2005.11.006.
[42]	Y. B. Yuan, Canonical duality solution for alternating support vector machine, Journal of Industrial and Management Optimization, 8 (2012), 611-621. doi: 10.3934/jimo.2012.8.611.
[43]	X. Zhang, et al., Discriminative locality preserving canonical correlation analysis, Pattern Recognition, Springer Berlin Heidelberg, 321 (2012), 341-349. doi: 10.1007/978-3-642-33506-8_43.
[44]	UCI, UCI Repository of machine learning databases, http://archive.ics.uci.edu/ml/.

show all references

References:

[1]	B. Abraham, et al., Dimensionality reduction approach to multivariate prediction, Comput Stat Data Anal, 48 (2005), 5-16. doi: 10.1016/j.csda.2003.11.021.
[2]	N. E. Ayat, et al., Automatic model selection for the optimization of SVM kernels, Pattern Recognition, 38 (2005), 1733-1745. doi: 10.1016/j.patcog.2005.03.011.
[3]	P. N. Belhumeur, et al., Eigenfaces vs. fisherfaces: Recognition using class specific linear projection, IEEE Transactions on Pattern Analysis and Machine Intelligence, 19 (1997), 711-720. doi: 10.1109/34.598228.
[4]	Q. N. Chen, et al., Hierarchical multi-view fisher discriminant analysis, International Conference on Neural Information Processing, 5864 (2009), 289-298. doi: 10.1007/978-3-642-10684-2_32.
[5]	T. Diethe, et al., Constructing nonlinear discriminants from multiple data views, Machine Learning and Knowledge Discovery in Databases, 6321 (2010), 328-343. doi: 10.1007/978-3-642-15880-3_27.
[6]	J. H. Friedman, et al., Regularized discriminant analysis, Journal of the American Statistics Association, 84 (1989), 165-175. doi: 10.1080/01621459.1989.10478752.
[7]	D. R. Hardoon, Sparse canonical correlation analysis, Machine Learning, 83 (2011), 331-353. doi: 10.1007/s10994-010-5222-7.
[8]	D. R. Hardoon, et al., Canonical correlation analysis: An overview with application to learning methods, Neural Computation, 16 (2004), 2639-2664. doi: 10.1162/0899766042321814.
[9]	X. He, et al., Local Preserving Projections, Advances in Neural Information Processing Systems, MIT Press, 2003.
[10]	X. He, et al., Face recognition using Laplicianfaces, IEEE Transactions on Pattern Analysis and Machine Intelligence, 27 (2005), 328-340.
[11]	L. Hoegaerts, et al., Subset based least squares subspace regression in RKHS, Neurocomputing, 63 (2005), 293-323. doi: 10.1016/j.neucom.2004.04.013.
[12]	Y. J. Huang, et al., Protein NMR recall, precision, and F-measure scores (RPF scores): Structure quality assessment measures based on information retrieval statistics, Journal of the American Chemical Society, 127 (2005), 1665-1674.
[13]	H. Hotelling, Relations between two sets of variates, Biometrika, 28 (1936), 312-377. doi: 10.2307/2333955.
[14]	Z. Ji, et al., Rank canonical correlation analysis and its application in visual search reranking, Signal Processing, 93 (2013), 2352-2360. doi: 10.1016/j.sigpro.2012.05.006.
[15]	X. Y. Jing, et al., Color image canonical correlation analysis for face feature extraction and recognition, Signal Processing, 91 (2011), 2132-2140. doi: 10.1016/j.sigpro.2011.02.016.
[16]	E. Kitani, et al., Um Tutorial sobre Analise de Componentes Principais para o Reconhecimento Automatico de Faces [R/OL], http://www.fei.edu.br/~cet, 2006.
[17]	P. L. Lai, et al., Kernel and nonlinear canonical correlation analysis, International Journal of Neural Systems, 10 (2000), 365-377. doi: 10.1016/S0129-0657(00)00034-X.
[18]	Y. Liu, et al., A survey of content-based image retrieval with high-level semantics, Pattern Recognition, 40 (2007), 262-282. doi: 10.1016/j.patcog.2006.04.045.
[19]	C. D. Manning, et al., Introduction to Information Retrieval, Cambridge: Cambridge university press, 2008. doi: 10.1017/CBO9780511809071.
[20]	T. Melzer, et al., Appearance models based on kernel canonical correlation analysis, Pattern recognition, 36 (2003), 1961-1971. doi: 10.1016/S0031-3203(03)00058-X.
[21]	T. Melzer, et al., Appearance models based on kernel canonical correlation analysis, Pattern Recognition, 36 (2003), 1961-1971. doi: 10.1016/S0031-3203(03)00058-X.
[22]	A. A. Nielsen, et al., Multiset canonical correlations analysis and multispectral truly multitemporal remote sensing data, IEEE Transactions on Image Processing, 11 (2002), 293-305. doi: 10.1109/83.988962.
[23]	S. Roweis, et al., Nonlinear dimensionality reduction by local linear embedding, Science, 290 (2000), 2323-2326. doi: 10.1126/science.290.5500.2323.
[24]	M. Ortega, et al., Supporting ranked boolean similarity queries in MARS, IEEE Transaction on Knowledge and Data Engineering, 10 (1998), 905-925. doi: 10.1109/69.738357.
[25]	N. Otopal, et al., Restricted kernel canonical correlation analysis, Linear Algebra and its Applications, 437 (2012), 1-13. doi: 10.1016/j.laa.2012.02.014.
[26]	O. A. B. Penatti, et al., Comparative study of global color and texture descriptors for web image retrieval, Journal of Visual Communication and Image Representation, 23 (2012), 359-380. doi: 10.1016/j.jvcir.2011.11.002.
[27]	Y. Peng, et al., Semi-supervised kernel canonical correlation analysis, Journal of Software, 19 (2008), 2822-2832.
[28]	Y. Peng, et al., A new canonical correlation analysis algorithm with local discrimination, Neural Processing Letters, 31 (2010), 1-15. doi: 10.1007/s11063-009-9123-3.
[29]	R. Pless, et al., A Survey of Manifold Learning, PIPSJ Transactions on Computer Vision and Applications, 1 (2009), 83-94.
[30]	F. S. Samaria, et al., Parameterisation of a stochastic model for human face identification, In Second IEEE Workshop on Applications of Computer Vision, (1994), 138-142. doi: 10.1109/ACV.1994.341300.
[31]	A. Sharma, et al., Generalized Multiview Analysis: A discriminative latent space, IEEE Conference on Computer Vision and Pattern Recognition, (2012), 2160-2167. doi: 10.1109/CVPR.2012.6247923.
[32]	L. Sun, et al., Canonical correlation analysis for multilabel classification: A least-squares formulation, extensions, and analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence, 33 (2011), 194-200.
[33]	Q. Sun, et al., A new method of feature fusion and its application in image recognition, Pattern Recognition, 38 (2005), 2437-2448. doi: 10.1016/j.patcog.2004.12.013.
[34]	T. K. Sun, et al., A novel method of combined feature extraction for recognition, IEEE Conference on Data Mining, (2008), 1043-1048. doi: 10.1109/ICDM.2008.28.
[35]	T. K. Sun, et al., Locality preserving CCA with applications to data visualization and pose estimation, Image and Vision Computing, 25 (2007), 531-543. doi: 10.1016/j.imavis.2006.04.014.
[36]	M. Turk, et al., Eigenfaces for recognition, Journal of Cognitive Neuroscience, 3 (1991), 71-86. doi: 10.1162/jocn.1991.3.1.71.
[37]	N. Vlassis, et al., Supervised linear feature extraction for mobile robot localization, Proceedings of the IEEE international conference on robotics and automation, 3 (2000), 2979-2984. doi: 10.1109/ROBOT.2000.846480.
[38]	Y. H. Yan, et al., A novel multiset integrated canonical correlation analysis framework and its application in feature fusion, Pattern Recognition, 44 (2011), 1031-1040. doi: 10.1016/j.patcog.2010.11.004.
[39]	X. Zhu, et al., Dimensionality reduction by mixed kernel canonical correlation analysis, Pattern Recognition, 45 (2012), 3003-3016. doi: 10.1016/j.patcog.2012.02.007.
[40]	J. Yang, et al., Feature fusion: Parallel strategy vs. serial strategy, Pattern Recognition, 36 (2003), 1369-1381. doi: 10.1016/S0031-3203(02)00262-5.
[41]	W. W. Yu, et al., Face recognition using discriminant locality preserving projections, Image and Vision computing, 24 (2006), 239-248. doi: 10.1016/j.imavis.2005.11.006.
[42]	Y. B. Yuan, Canonical duality solution for alternating support vector machine, Journal of Industrial and Management Optimization, 8 (2012), 611-621. doi: 10.3934/jimo.2012.8.611.
[43]	X. Zhang, et al., Discriminative locality preserving canonical correlation analysis, Pattern Recognition, Springer Berlin Heidelberg, 321 (2012), 341-349. doi: 10.1007/978-3-642-33506-8_43.
[44]	UCI, UCI Repository of machine learning databases, http://archive.ics.uci.edu/ml/.

[1]	Jia Cai, Junyi Huo. Sparse generalized canonical correlation analysis via linearized Bregman method. Communications on Pure and Applied Analysis, 2020, 19 (8) : 3933-3945. doi: 10.3934/cpaa.2020173
[2]	Sohana Jahan. Discriminant analysis of regularized multidimensional scaling. Numerical Algebra, Control and Optimization, 2021, 11 (2) : 255-267. doi: 10.3934/naco.2020024
[3]	Wenjuan Jia, Yingjie Deng, Chenyang Xin, Xiaodong Liu, Witold Pedrycz. A classification algorithm with Linear Discriminant Analysis and Axiomatic Fuzzy Sets. Mathematical Foundations of Computing, 2019, 2 (1) : 73-81. doi: 10.3934/mfc.2019006
[4]	Jian Luo, Shu-Cherng Fang, Yanqin Bai, Zhibin Deng. Fuzzy quadratic surface support vector machine based on fisher discriminant analysis. Journal of Industrial and Management Optimization, 2016, 12 (1) : 357-373. doi: 10.3934/jimo.2016.12.357
[5]	Boling Guo, Guoli Zhou. Finite dimensionality of global attractor for the solutions to 3D viscous primitive equations of large-scale moist atmosphere. Discrete and Continuous Dynamical Systems - B, 2018, 23 (10) : 4305-4327. doi: 10.3934/dcdsb.2018160
[6]	Fanwen Meng, Kiok Liang Teow, Kelvin Wee Sheng Teo, Chee Kheong Ooi, Seow Yian Tay. Predicting 72-hour reattendance in emergency departments using discriminant analysis via mixed integer programming with electronic medical records. Journal of Industrial and Management Optimization, 2019, 15 (2) : 947-962. doi: 10.3934/jimo.2018079
[7]	Rentsen Enkhbat, Evgeniya A. Finkelstein, Anton S. Anikin, Alexandr Yu. Gornov. Global optimization reduction of generalized Malfatti's problem. Numerical Algebra, Control and Optimization, 2017, 7 (2) : 211-221. doi: 10.3934/naco.2017015
[8]	Adrian Viorel, Cristian D. Alecsa, Titus O. Pinţa. Asymptotic analysis of a structure-preserving integrator for damped Hamiltonian systems. Discrete and Continuous Dynamical Systems, 2021, 41 (7) : 3319-3341. doi: 10.3934/dcds.2020407
[9]	Palash Sarkar, Subhadip Singha. Classical reduction of gap SVP to LWE: A concrete security analysis. Advances in Mathematics of Communications, 2021 doi: 10.3934/amc.2021004
[10]	Claude Carlet, Khoongming Khoo, Chu-Wee Lim, Chuan-Wen Loe. On an improved correlation analysis of stream ciphers using multi-output Boolean functions and the related generalized notion of nonlinearity. Advances in Mathematics of Communications, 2008, 2 (2) : 201-221. doi: 10.3934/amc.2008.2.201
[11]	Lassi Roininen, Markku S. Lehtinen, Sari Lasanen, Mikko Orispää, Markku Markkanen. Correlation priors. Inverse Problems and Imaging, 2011, 5 (1) : 167-184. doi: 10.3934/ipi.2011.5.167
[12]	Hayato Chiba, Georgi S. Medvedev. The mean field analysis of the kuramoto model on graphs Ⅱ. asymptotic stability of the incoherent state, center manifold reduction, and bifurcations. Discrete and Continuous Dynamical Systems, 2019, 39 (7) : 3897-3921. doi: 10.3934/dcds.2019157
[13]	Marc Massot. Singular perturbation analysis for the reduction of complex chemistry in gaseous mixtures using the entropic structure. Discrete and Continuous Dynamical Systems - B, 2002, 2 (3) : 433-456. doi: 10.3934/dcdsb.2002.2.433
[14]	Georg Vossen, Stefan Volkwein. Model reduction techniques with a-posteriori error analysis for linear-quadratic optimal control problems. Numerical Algebra, Control and Optimization, 2012, 2 (3) : 465-485. doi: 10.3934/naco.2012.2.465
[15]	Jan-Phillip Bäcker, Matthias Röger. Analysis and asymptotic reduction of a bulk-surface reaction-diffusion model of Gierer-Meinhardt type. Communications on Pure and Applied Analysis, 2022, 21 (4) : 1139-1155. doi: 10.3934/cpaa.2022013
[16]	Marilena N. Poulou, Nikolaos M. Stavrakakis. Finite dimensionality of a Klein-Gordon-Schrödinger type system. Discrete and Continuous Dynamical Systems - S, 2009, 2 (1) : 149-161. doi: 10.3934/dcdss.2009.2.149
[17]	Alexander Sakhnovich. Dynamical canonical systems and their explicit solutions. Discrete and Continuous Dynamical Systems, 2017, 37 (3) : 1679-1689. doi: 10.3934/dcds.2017069
[18]	Nikita Selinger. Topological characterization of canonical Thurston obstructions. Journal of Modern Dynamics, 2013, 7 (1) : 99-117. doi: 10.3934/jmd.2013.7.99
[19]	Peter Scott and Gadde A. Swarup. Regular neighbourhoods and canonical decompositions for groups. Electronic Research Announcements, 2002, 8: 20-28.
[20]	Vladimir Lazić, Fanjun Meng. On Nonvanishing for uniruled log canonical pairs. Electronic Research Archive, 2021, 29 (5) : 3297-3308. doi: 10.3934/era.2021039

2021 Impact Factor: 1.411

Tools

Metrics

Other articles
by authors

on AIMS
Yubo Yuan Chenglong Ma Dongmei Pu
on Google Scholar
Yubo Yuan Chenglong Ma Dongmei Pu

[Back to Top]