American Institute of Mathematical Sciences

Advanced
2011, 1(1): 151-169. doi: 10.3934/naco.2011.1.151

Performance evaluation of multiobjective multiclass support vector machines maximizing geometric margins

Keiji Tatsumi 1, , Masashi Akao 1, , Ryo Kawachi 1, and  Tetsuzo Tanino 1,

1. 

Graduate School of Engineering, Osaka University, Yamada-Oka 1-2, Suita, Osaka 565-0871, Japan, Japan, Japan, Japan

Received  September 2010 Revised  December 2010 Published  February 2011

The all-together method is one of the support vector machine (SVM) for multiclass classification by using a piece-wise linear function. Recently, we proposed a new hard-margin all-together model maximizing geometric margins in the sense of multiobjective optimization for the high generalization ability, which is called the multiobjective multiclass SVM (MMSVM). Moreover, we derived its solving techniques which can find a Pareto optimal solution for the MMSVM, and verified that classifiers with larger geometric margins were obtained by the proposed techniques through numerical experiments. However, the experiments are not enough to evaluate the classification performance of the proposed model, and the MMSVM is a hard-margin model which can be applied to only piecewise linearly separable data. Therefore, in this paper, we extend the hard-margin model into soft-margin one by introducing penalty functions for the slack margin variables, and derive a single-objective second-order cone programming (SOCP) problem to solve it. Furthermore, through numerical experiments we verify the classification performance of the hard and soft-margin MMSVMs for benchmark problems.
Keywords: multiobjective optimization, second-order cone programming., multiclass classification, Support vector machine.
Mathematics Subject Classification: Primary: 62H30, 68T10; Secondary: 90C2.
Citation: Keiji Tatsumi, Masashi Akao, Ryo Kawachi, Tetsuzo Tanino. Performance evaluation of multiobjective multiclass support vector machines maximizing geometric margins. Numerical Algebra, Control and Optimization, 2011, 1 (1) : 151-169. doi: 10.3934/naco.2011.1.151
References:
[1]

S. Abe, "Support Vector Machines for Pattern Classification," Springer-Verlag, New York, 2005.

[2]

F. Alizadeh and D. Goldfarb, Second-order cone programming, Mathematical Programming, Ser. B, 95 (2003), 3-51.

[3]

L. Bottou, C. Cortes, J. Denker, H. Drucker, I. Guyon, L. Jackel, Y. LeCun, U. Muller, E. Sackinger, P. Simard and V. Vapnik, Comparison of classifier methods: A case study in handwriting digit recognition, in "Proc. Int. Conf. Pattern Recognition,'' (1994), 77-87.

[4]

E. J. Bredensteiner and K. P. Bennett, Multicategory classification by support vector machines, Computational Optimization and Applications, 12 (1999), 53-79. doi: 10.1023/A:1008663629662.

[5]

M. Ehrgott, "Multicriteria Optimization,'' 2nd edition, Springer-Verlag, Berlin, 2005.

[6]

Y. Guermeur, Combining discriminant models with new multiclass SVMs, Neuro COLT2 Technical Report Series, 2000.

[7]

U. Kressel, Pairwise classification and support vector machines, in "Advances in kernel methods - Support vector learning'' (eds. B. Schölkopf, C. Burges, and A. J. Smola), MIT Press, Cambridge, (1999), 255-268.

[8]

C. W. Hsh and C. J. Lin, A comparison of methods for multiclass support vector machines, IEEE Trans. Neural Networks, 13 (2) (2002), 181-201.

[9]

H. D. Mittelmann, An independent benchmarking of SDP and SOCP solvers, Mathematical Programming, Ser. B, 95 (2003), 407-430.

[10]

K. R. Müller, S. Mika, G. Rätsch, K. Tsuda and B. Shölkopf, An introduction to kernel-based learning algorithms, IEEE Trans. Neural Networks, 12 (2), (2001), 181-201. doi: 10.1109/72.914517.

[11]

A. Passerini, M. Pontil and P. Frasconi, New results on error correcting output codes of kernel machines, IEEE Trans. Neural Networks, 14 (1), (2004), 45-54. doi: 10.1109/TNN.2003.820841.

[12]

J. C. Platt, N. Cristianini and J. Shawe-Taylor, Large margin DAG's for multiclass classification, in "Advances in Neural Information Processing Systems,'' Cambridge, MA: MIT Press, 12 (2000), 547-553.

[13]

K. Tatsumi, K. Hayashida, H. Higashi and T. Tanino, Multi-objective multiclass support vector machine for pattern recognition, in "Proc. SICE Annual Conference 2007,'' 1095-1098. doi: 10.1109/SICE.2007.4421147.

[14]

K. Tatsumi, R. Kawachi, K. Hayashida and T. Tanino, Multiobjective multiclass support vector machines maximizing geometric margins, Pacific Journal of Optimization, 6 (1), (2000), 115-140.

[15]

J. S. Taylor and N. Cristianini, "Kernel Methods for Pattern Analysis,'' Cambridge University Press, 2004.

[16]

, UCI benchmark repository of artificial and real data sets, University of California Irvine, Available from: http://www.ics.uci.edu/~mlearn/databases/

[17]

J. Weston and C. Watkins, Multi-class support vector machines, Technical report CSD-TR-98-04, Univ. London, Royal Holloway, (1998).

[18]

V. N. Vapnik, "Statistical Learning Theory,'' A Wiley-Interscience Publication, 1998.

show all references

References:
[1]

S. Abe, "Support Vector Machines for Pattern Classification," Springer-Verlag, New York, 2005.

[2]

F. Alizadeh and D. Goldfarb, Second-order cone programming, Mathematical Programming, Ser. B, 95 (2003), 3-51.

[3]

L. Bottou, C. Cortes, J. Denker, H. Drucker, I. Guyon, L. Jackel, Y. LeCun, U. Muller, E. Sackinger, P. Simard and V. Vapnik, Comparison of classifier methods: A case study in handwriting digit recognition, in "Proc. Int. Conf. Pattern Recognition,'' (1994), 77-87.

[4]

E. J. Bredensteiner and K. P. Bennett, Multicategory classification by support vector machines, Computational Optimization and Applications, 12 (1999), 53-79. doi: 10.1023/A:1008663629662.

[5]

M. Ehrgott, "Multicriteria Optimization,'' 2nd edition, Springer-Verlag, Berlin, 2005.

[6]

Y. Guermeur, Combining discriminant models with new multiclass SVMs, Neuro COLT2 Technical Report Series, 2000.

[7]

U. Kressel, Pairwise classification and support vector machines, in "Advances in kernel methods - Support vector learning'' (eds. B. Schölkopf, C. Burges, and A. J. Smola), MIT Press, Cambridge, (1999), 255-268.

[8]

C. W. Hsh and C. J. Lin, A comparison of methods for multiclass support vector machines, IEEE Trans. Neural Networks, 13 (2) (2002), 181-201.

[9]

H. D. Mittelmann, An independent benchmarking of SDP and SOCP solvers, Mathematical Programming, Ser. B, 95 (2003), 407-430.

[10]

K. R. Müller, S. Mika, G. Rätsch, K. Tsuda and B. Shölkopf, An introduction to kernel-based learning algorithms, IEEE Trans. Neural Networks, 12 (2), (2001), 181-201. doi: 10.1109/72.914517.

[11]

A. Passerini, M. Pontil and P. Frasconi, New results on error correcting output codes of kernel machines, IEEE Trans. Neural Networks, 14 (1), (2004), 45-54. doi: 10.1109/TNN.2003.820841.

[12]

J. C. Platt, N. Cristianini and J. Shawe-Taylor, Large margin DAG's for multiclass classification, in "Advances in Neural Information Processing Systems,'' Cambridge, MA: MIT Press, 12 (2000), 547-553.

[13]

K. Tatsumi, K. Hayashida, H. Higashi and T. Tanino, Multi-objective multiclass support vector machine for pattern recognition, in "Proc. SICE Annual Conference 2007,'' 1095-1098. doi: 10.1109/SICE.2007.4421147.

[14]

K. Tatsumi, R. Kawachi, K. Hayashida and T. Tanino, Multiobjective multiclass support vector machines maximizing geometric margins, Pacific Journal of Optimization, 6 (1), (2000), 115-140.

[15]

J. S. Taylor and N. Cristianini, "Kernel Methods for Pattern Analysis,'' Cambridge University Press, 2004.

[16]

, UCI benchmark repository of artificial and real data sets, University of California Irvine, Available from: http://www.ics.uci.edu/~mlearn/databases/

[17]

J. Weston and C. Watkins, Multi-class support vector machines, Technical report CSD-TR-98-04, Univ. London, Royal Holloway, (1998).

[18]

V. N. Vapnik, "Statistical Learning Theory,'' A Wiley-Interscience Publication, 1998.

[1]

Yi Zhang, Yong Jiang, Liwei Zhang, Jiangzhong Zhang. A perturbation approach for an inverse linear second-order cone programming. Journal of Industrial and Management Optimization, 2013, 9 (1) : 171-189. doi: 10.3934/jimo.2013.9.171
[2]

Yubo Yuan, Weiguo Fan, Dongmei Pu. Spline function smooth support vector machine for classification. Journal of Industrial and Management Optimization, 2007, 3 (3) : 529-542. doi: 10.3934/jimo.2007.3.529
[3]

Xiaoni Chi, Zhongping Wan, Zijun Hao. Second order sufficient conditions for a class of bilevel programs with lower level second-order cone programming problem. Journal of Industrial and Management Optimization, 2015, 11 (4) : 1111-1125. doi: 10.3934/jimo.2015.11.1111
[4]

Ye Tian, Shu-Cherng Fang, Zhibin Deng, Wenxun Xing. Computable representation of the cone of nonnegative quadratic forms over a general second-order cone and its application to completely positive programming. Journal of Industrial and Management Optimization, 2013, 9 (3) : 703-721. doi: 10.3934/jimo.2013.9.703
[5]

Liwei Zhang, Jihong Zhang, Yule Zhang. Second-order optimality conditions for cone constrained multi-objective optimization. Journal of Industrial and Management Optimization, 2018, 14 (3) : 1041-1054. doi: 10.3934/jimo.2017089
[6]

Lin Zhu, Xinzhen Zhang. Semidefinite relaxation method for polynomial optimization with second-order cone complementarity constraints. Journal of Industrial and Management Optimization, 2022, 18 (3) : 1505-1517. doi: 10.3934/jimo.2021030
[7]

Xinmin Yang. On second order symmetric duality in nondifferentiable multiobjective programming. Journal of Industrial and Management Optimization, 2009, 5 (4) : 697-703. doi: 10.3934/jimo.2009.5.697
[8]

Liping Tang, Xinmin Yang, Ying Gao. Higher-order symmetric duality for multiobjective programming with cone constraints. Journal of Industrial and Management Optimization, 2020, 16 (4) : 1873-1884. doi: 10.3934/jimo.2019033
[9]

Anurag Jayswala, Tadeusz Antczakb, Shalini Jha. Second order modified objective function method for twice differentiable vector optimization problems over cone constraints. Numerical Algebra, Control and Optimization, 2019, 9 (2) : 133-145. doi: 10.3934/naco.2019010
[10]

Qilin Wang, Shengji Li, Kok Lay Teo. Continuity of second-order adjacent derivatives for weak perturbation maps in vector optimization. Numerical Algebra, Control and Optimization, 2011, 1 (3) : 417-433. doi: 10.3934/naco.2011.1.417
[11]

Qianru Zhai, Ye Tian, Jingyue Zhou. A SMOTE-based quadratic surface support vector machine for imbalanced classification with mislabeled information. Journal of Industrial and Management Optimization, 2022  doi: 10.3934/jimo.2021230
[12]

Shiyun Wang, Yong-Jin Liu, Yong Jiang. A majorized penalty approach to inverse linear second order cone programming problems. Journal of Industrial and Management Optimization, 2014, 10 (3) : 965-976. doi: 10.3934/jimo.2014.10.965
[13]

Ning Lu, Ying Liu. Application of support vector machine model in wind power prediction based on particle swarm optimization. Discrete and Continuous Dynamical Systems - S, 2015, 8 (6) : 1267-1276. doi: 10.3934/dcdss.2015.8.1267
[14]

Yubo Yuan. Canonical duality solution for alternating support vector machine. Journal of Industrial and Management Optimization, 2012, 8 (3) : 611-621. doi: 10.3934/jimo.2012.8.611
[15]

Xi-De Zhu, Li-Ping Pang, Gui-Hua Lin. Two approaches for solving mathematical programs with second-order cone complementarity constraints. Journal of Industrial and Management Optimization, 2015, 11 (3) : 951-968. doi: 10.3934/jimo.2015.11.951
[16]

Yanhong Yuan, Hongwei Zhang, Liwei Zhang. A smoothing Newton method for generalized Nash equilibrium problems with second-order cone constraints. Numerical Algebra, Control and Optimization, 2012, 2 (1) : 1-18. doi: 10.3934/naco.2012.2.1
[17]

Li Chu, Bo Wang, Jie Zhang, Hong-Wei Zhang. Convergence analysis of a smoothing SAA method for a stochastic mathematical program with second-order cone complementarity constraints. Journal of Industrial and Management Optimization, 2021, 17 (4) : 1863-1886. doi: 10.3934/jimo.2020050
[18]

Xin-He Miao, Kai Yao, Ching-Yu Yang, Jein-Shan Chen. Levenberg-Marquardt method for absolute value equation associated with second-order cone. Numerical Algebra, Control and Optimization, 2022, 12 (1) : 47-61. doi: 10.3934/naco.2021050
[19]

Narges Torabi Golsefid, Maziar Salahi. Second order cone programming formulation of the fixed cost allocation in DEA based on Nash bargaining game. Numerical Algebra, Control and Optimization, 2021  doi: 10.3934/naco.2021032
[20]

Lijun Zhang, Chaudry Masood Khalique. Classification and bifurcation of a class of second-order ODEs and its application to nonlinear PDEs. Discrete and Continuous Dynamical Systems - S, 2018, 11 (4) : 759-772. doi: 10.3934/dcdss.2018048

 Impact Factor: 

Tools

Metrics

  • PDF downloads (76)
  • HTML views (0)
  • Cited by (2)

Other articles
by authors

[Back to Top]

Copyright © 2022 American Institute of Mathematical Sciences | Privacy Policy