
-
Previous Article
AIMS: Average information matrix splitting
- MFC Home
- This Issue
-
Next Article
Modeling interactive components by coordinate kernel polynomial models
Support vector machine classifiers by non-Euclidean margins
School of Mathematical Sciences, South China Normal University, Guangzhou, Guangdong 510631, China |
In this article, the classical support vector machine (SVM) classifiers are generalized by the non-Euclidean margins. We first extend the linear models of the SVM classifiers by the non-Euclidean margins including the theorems and algorithms of the SVM classifiers by the hard margins and the soft margins. Specially, the SVM classifiers by the $ \infty $-norm margins can be solved by the 1-norm optimization with sparsity. Next, we show that the non-linear models of the SVM classifiers by the $ q $-norm margins can be equivalently transferred to the SVM in the $ p $-norm reproducing kernel Banach spaces given by the hinge loss, where $ 1/p+1/q = 1 $. Finally, we illustrate the numerical examples of artificial data and real data to compare the different algorithms of the SVM classifiers by the $ \infty $-norm margin.
References:
[1] |
B. E. Boser, I. M. Guyon and V. N. Vapnik, A training algorithm for optimal margin classifiers, in Proceedings of the Fifth Annual Workshop on Computational learning theory, ACM, (1992), 144–152.
doi: 10.1145/130385.130401. |
[2] |
P. Bühlmann and S. van de Geer, Statistics for High-Dimensional Data. Methods, Theory and Applications, Springer Series in Statistics, Springer, Heidelberg, 2011.
doi: 10.1007/978-3-642-20192-9. |
[3] |
L. Chen and H. Zhang,
Statistical margin error bounds for l1-norm support vector machines, Neurocomputing, 339 (2019), 210-216.
doi: 10.1016/j.neucom.2019.02.015. |
[4] |
C. Cortes and V. Vapnik,
Support-vector networks, Machine Learning, 20 (1995), 273-297.
doi: 10.1007/BF00994018. |
[5] |
R. Der and D. Lee, Large-margin classification in banach spaces, Journal of Machine Learning Research - Proceedings Track, 2 (2007), 91-98. Google Scholar |
[6] |
I. Ekeland and T. Turnbull, Infinite-Dimensional Optimization and Convexity, Chicago Lectures in Mathematics, University of Chicago Press, Chicago, IL, 1983.
![]() |
[7] |
T. Hastie, R. Tibshirani and M. Wainwright, Statistical Learning with Sparsity: The Lasso and Generalizations, Monographs on Statistics and Applied Probability, 143. CRC Press, Boca Raton, FL, 2015.
![]() |
[8] |
L. Huang, C. Liu, L. Tan and Q. Ye, Generalized representer theorems in Banach spaces, Anal. Appl. (Singap.), (2019).
doi: 10.1142/S0219530519410100. |
[9] |
O. L. Mangasarian,
Arbitrary-norm separating plane, Operations Research Letters, 24 (1999), 15-23.
doi: 10.1016/S0167-6377(98)00049-2. |
[10] |
J. Platt, Sequential minimal optimization: A fast algorithm for training support vector machines., Google Scholar |
[11] |
L. Q. Qi, H. B. Chen and Y. N.Chen, Tensor Eigenvalues and Their Applications, Advances in Mechanics and Mathematics, 39. Springer, Singapore, 2018.
doi: 10.1007/978-981-10-8058-6. |
[12] |
R. T. Rockafellar, Convex Analysis, Princeton Mathematical Series, No. 28, Princeton University Press, Princeton, N.J., 1970.
![]() |
[13] | B. Schölkopf and A. J. Smola, Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond, The MIT Press, Cambridge, 2001. Google Scholar |
[14] |
G. H. Song and H. Z. Zhang,
Reproducing kernel Banach spaces with the $\ell^1$ norm Ⅱ: Error analysis for regularized least square regression, Neural Comput., 23 (2011), 2713-2729.
doi: 10.1162/NECO_a_00178. |
[15] |
G. H. Song, H. Z. Zhang and F. J. Hickernell,
Reproducing kernel Banach spaces with the $\ell^1$ norm, Appl. Comput. Harmon. Anal., 34 (2013), 96-116.
doi: 10.1016/j.acha.2012.03.009. |
[16] |
I. Steinwart and A. Christmann, Support Vector Machines, Information Science and Statistics, Springer, New York, 2008. |
[17] |
V. N. Vapnik, The Nature of Statistical Learning Theory, Springer-Verlag, New York, 1995.
doi: 10.1007/978-1-4757-2440-0. |
[18] |
Y. S. Xu and Q. Ye, Generalized mercer kernels and reproducing kernel banach spaces, Mem. Amer. Math. Soc., 258 (2019).
doi: 10.1090/memo/1243. |
[19] |
H. Yang, X. Yang, F. Zhang, Q. Ye and X. Fan,
Infinite norm large margin classifier, International Journal of Machine Learning and Cybernetics, 10 (2019), 2449-2457.
doi: 10.1007/s13042-018-0881-y. |
[20] |
H. Z. Zhang, Y. S. Xu and J. Zhang,
Reproducing kernel Banach spaces for machine learning, J. Mach. Learn. Res., 10 (2009), 2741-2775.
doi: 10.1109/IJCNN.2009.5179093. |
[21] |
L. Zhang and W. Zhou,
On the sparseness of 1-norm support vector machines, Neural Networks, 23 (2010), 373-385.
doi: 10.1016/j.neunet.2009.11.012. |
[22] |
J. Zhu, S. Rosset, R. Tibshirani and T. J. Hastie, 1-norm support vector machines, in Advances in Neural Information Processing Systems, (2004), 49–56. Google Scholar |
show all references
References:
[1] |
B. E. Boser, I. M. Guyon and V. N. Vapnik, A training algorithm for optimal margin classifiers, in Proceedings of the Fifth Annual Workshop on Computational learning theory, ACM, (1992), 144–152.
doi: 10.1145/130385.130401. |
[2] |
P. Bühlmann and S. van de Geer, Statistics for High-Dimensional Data. Methods, Theory and Applications, Springer Series in Statistics, Springer, Heidelberg, 2011.
doi: 10.1007/978-3-642-20192-9. |
[3] |
L. Chen and H. Zhang,
Statistical margin error bounds for l1-norm support vector machines, Neurocomputing, 339 (2019), 210-216.
doi: 10.1016/j.neucom.2019.02.015. |
[4] |
C. Cortes and V. Vapnik,
Support-vector networks, Machine Learning, 20 (1995), 273-297.
doi: 10.1007/BF00994018. |
[5] |
R. Der and D. Lee, Large-margin classification in banach spaces, Journal of Machine Learning Research - Proceedings Track, 2 (2007), 91-98. Google Scholar |
[6] |
I. Ekeland and T. Turnbull, Infinite-Dimensional Optimization and Convexity, Chicago Lectures in Mathematics, University of Chicago Press, Chicago, IL, 1983.
![]() |
[7] |
T. Hastie, R. Tibshirani and M. Wainwright, Statistical Learning with Sparsity: The Lasso and Generalizations, Monographs on Statistics and Applied Probability, 143. CRC Press, Boca Raton, FL, 2015.
![]() |
[8] |
L. Huang, C. Liu, L. Tan and Q. Ye, Generalized representer theorems in Banach spaces, Anal. Appl. (Singap.), (2019).
doi: 10.1142/S0219530519410100. |
[9] |
O. L. Mangasarian,
Arbitrary-norm separating plane, Operations Research Letters, 24 (1999), 15-23.
doi: 10.1016/S0167-6377(98)00049-2. |
[10] |
J. Platt, Sequential minimal optimization: A fast algorithm for training support vector machines., Google Scholar |
[11] |
L. Q. Qi, H. B. Chen and Y. N.Chen, Tensor Eigenvalues and Their Applications, Advances in Mechanics and Mathematics, 39. Springer, Singapore, 2018.
doi: 10.1007/978-981-10-8058-6. |
[12] |
R. T. Rockafellar, Convex Analysis, Princeton Mathematical Series, No. 28, Princeton University Press, Princeton, N.J., 1970.
![]() |
[13] | B. Schölkopf and A. J. Smola, Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond, The MIT Press, Cambridge, 2001. Google Scholar |
[14] |
G. H. Song and H. Z. Zhang,
Reproducing kernel Banach spaces with the $\ell^1$ norm Ⅱ: Error analysis for regularized least square regression, Neural Comput., 23 (2011), 2713-2729.
doi: 10.1162/NECO_a_00178. |
[15] |
G. H. Song, H. Z. Zhang and F. J. Hickernell,
Reproducing kernel Banach spaces with the $\ell^1$ norm, Appl. Comput. Harmon. Anal., 34 (2013), 96-116.
doi: 10.1016/j.acha.2012.03.009. |
[16] |
I. Steinwart and A. Christmann, Support Vector Machines, Information Science and Statistics, Springer, New York, 2008. |
[17] |
V. N. Vapnik, The Nature of Statistical Learning Theory, Springer-Verlag, New York, 1995.
doi: 10.1007/978-1-4757-2440-0. |
[18] |
Y. S. Xu and Q. Ye, Generalized mercer kernels and reproducing kernel banach spaces, Mem. Amer. Math. Soc., 258 (2019).
doi: 10.1090/memo/1243. |
[19] |
H. Yang, X. Yang, F. Zhang, Q. Ye and X. Fan,
Infinite norm large margin classifier, International Journal of Machine Learning and Cybernetics, 10 (2019), 2449-2457.
doi: 10.1007/s13042-018-0881-y. |
[20] |
H. Z. Zhang, Y. S. Xu and J. Zhang,
Reproducing kernel Banach spaces for machine learning, J. Mach. Learn. Res., 10 (2009), 2741-2775.
doi: 10.1109/IJCNN.2009.5179093. |
[21] |
L. Zhang and W. Zhou,
On the sparseness of 1-norm support vector machines, Neural Networks, 23 (2010), 373-385.
doi: 10.1016/j.neunet.2009.11.012. |
[22] |
J. Zhu, S. Rosset, R. Tibshirani and T. J. Hastie, 1-norm support vector machines, in Advances in Neural Information Processing Systems, (2004), 49–56. Google Scholar |












MNIST | Training Errors | Test Errors | Sparsity |
Linear SVM classifier by ∞-norm margin | 0/9939 | 5/1967 | 654/785 |
Kernel SVM classifier by ∞-norm margin | 0/9939 | 4/1967 | 1760/9939 |
MNIST | Training Errors | Test Errors | Sparsity |
Linear SVM classifier by ∞-norm margin | 0/9939 | 5/1967 | 654/785 |
Kernel SVM classifier by ∞-norm margin | 0/9939 | 4/1967 | 1760/9939 |
Handwritten Alphabets | Training Errors | Test Errors | Sparsity |
Linear SVM> classifier by ∞-norm margin | 0/7000 | 69/3000 | 401/785 |
Kernel SVM classifier by ∞-norm margin | 0/7000 | 14/3000 | 1339/7000 |
Handwritten Alphabets | Training Errors | Test Errors | Sparsity |
Linear SVM> classifier by ∞-norm margin | 0/7000 | 69/3000 | 401/785 |
Kernel SVM classifier by ∞-norm margin | 0/7000 | 14/3000 | 1339/7000 |
[1] |
Liam Burrows, Weihong Guo, Ke Chen, Francesco Torella. Reproducible kernel Hilbert space based global and local image segmentation. Inverse Problems & Imaging, 2021, 15 (1) : 1-25. doi: 10.3934/ipi.2020048 |
[2] |
Jianhua Huang, Yanbin Tang, Ming Wang. Singular support of the global attractor for a damped BBM equation. Discrete & Continuous Dynamical Systems - B, 2020 doi: 10.3934/dcdsb.2020345 |
[3] |
Xin Guo, Lexin Li, Qiang Wu. Modeling interactive components by coordinate kernel polynomial models. Mathematical Foundations of Computing, 2020, 3 (4) : 263-277. doi: 10.3934/mfc.2020010 |
[4] |
Min Ji, Xinna Ye, Fangyao Qian, T.C.E. Cheng, Yiwei Jiang. Parallel-machine scheduling in shared manufacturing. Journal of Industrial & Management Optimization, 2020 doi: 10.3934/jimo.2020174 |
[5] |
Bahaaeldin Abdalla, Thabet Abdeljawad. Oscillation criteria for kernel function dependent fractional dynamic equations. Discrete & Continuous Dynamical Systems - S, 2020 doi: 10.3934/dcdss.2020443 |
[6] |
Sumit Arora, Manil T. Mohan, Jaydev Dabas. Approximate controllability of a Sobolev type impulsive functional evolution system in Banach spaces. Mathematical Control & Related Fields, 2020 doi: 10.3934/mcrf.2020049 |
[7] |
Dmitry Dolgopyat. The work of Sébastien Gouëzel on limit theorems and on weighted Banach spaces. Journal of Modern Dynamics, 2020, 16: 351-371. doi: 10.3934/jmd.2020014 |
[8] |
Manxue You, Shengjie Li. Perturbation of Image and conjugate duality for vector optimization. Journal of Industrial & Management Optimization, 2020 doi: 10.3934/jimo.2020176 |
[9] |
Mostafa Mbekhta. Representation and approximation of the polar factor of an operator on a Hilbert space. Discrete & Continuous Dynamical Systems - S, 2020 doi: 10.3934/dcdss.2020463 |
[10] |
Wen Li, Wei-Hui Liu, Seak Weng Vong. Perron vector analysis for irreducible nonnegative tensors and its applications. Journal of Industrial & Management Optimization, 2021, 17 (1) : 29-50. doi: 10.3934/jimo.2019097 |
[11] |
Liping Tang, Ying Gao. Some properties of nonconvex oriented distance function and applications to vector optimization problems. Journal of Industrial & Management Optimization, 2021, 17 (1) : 485-500. doi: 10.3934/jimo.2020117 |
[12] |
Soniya Singh, Sumit Arora, Manil T. Mohan, Jaydev Dabas. Approximate controllability of second order impulsive systems with state-dependent delay in Banach spaces. Evolution Equations & Control Theory, 2020 doi: 10.3934/eect.2020103 |
[13] |
Kengo Nakai, Yoshitaka Saiki. Machine-learning construction of a model for a macroscopic fluid variable using the delay-coordinate of a scalar observable. Discrete & Continuous Dynamical Systems - S, 2021, 14 (3) : 1079-1092. doi: 10.3934/dcdss.2020352 |
[14] |
Hirokazu Ninomiya. Entire solutions of the Allen–Cahn–Nagumo equation in a multi-dimensional space. Discrete & Continuous Dynamical Systems - A, 2021, 41 (1) : 395-412. doi: 10.3934/dcds.2020364 |
[15] |
Dong-Ho Tsai, Chia-Hsing Nien. On space-time periodic solutions of the one-dimensional heat equation. Discrete & Continuous Dynamical Systems - A, 2020, 40 (6) : 3997-4017. doi: 10.3934/dcds.2020037 |
[16] |
Petr Čoupek, María J. Garrido-Atienza. Bilinear equations in Hilbert space driven by paths of low regularity. Discrete & Continuous Dynamical Systems - B, 2021, 26 (1) : 121-154. doi: 10.3934/dcdsb.2020230 |
[17] |
Boris Andreianov, Mohamed Maliki. On classes of well-posedness for quasilinear diffusion equations in the whole space. Discrete & Continuous Dynamical Systems - S, 2021, 14 (2) : 505-531. doi: 10.3934/dcdss.2020361 |
[18] |
Jianfeng Huang, Haihua Liang. Limit cycles of planar system defined by the sum of two quasi-homogeneous vector fields. Discrete & Continuous Dynamical Systems - B, 2021, 26 (2) : 861-873. doi: 10.3934/dcdsb.2020145 |
[19] |
Jingjing Wang, Zaiyun Peng, Zhi Lin, Daqiong Zhou. On the stability of solutions for the generalized vector quasi-equilibrium problems via free-disposal set. Journal of Industrial & Management Optimization, 2021, 17 (2) : 869-887. doi: 10.3934/jimo.2020002 |
[20] |
Abdollah Borhanifar, Maria Alessandra Ragusa, Sohrab Valizadeh. High-order numerical method for two-dimensional Riesz space fractional advection-dispersion equation. Discrete & Continuous Dynamical Systems - B, 2020 doi: 10.3934/dcdsb.2020355 |
Impact Factor:
Tools
Metrics
Other articles
by authors
[Back to Top]