American Institute of Mathematical Sciences

Advanced
2011, 1(1): 151-169. doi: 10.3934/naco.2011.1.151

Performance evaluation of multiobjective multiclass support vector machines maximizing geometric margins

Keiji Tatsumi 1, , Masashi Akao 1, , Ryo Kawachi 1, and  Tetsuzo Tanino 1,

1. 

Graduate School of Engineering, Osaka University, Yamada-Oka 1-2, Suita, Osaka 565-0871, Japan, Japan, Japan, Japan

Received  September 2010 Revised  December 2010 Published  February 2011

The all-together method is one of the support vector machine (SVM) for multiclass classification by using a piece-wise linear function. Recently, we proposed a new hard-margin all-together model maximizing geometric margins in the sense of multiobjective optimization for the high generalization ability, which is called the multiobjective multiclass SVM (MMSVM). Moreover, we derived its solving techniques which can find a Pareto optimal solution for the MMSVM, and verified that classifiers with larger geometric margins were obtained by the proposed techniques through numerical experiments. However, the experiments are not enough to evaluate the classification performance of the proposed model, and the MMSVM is a hard-margin model which can be applied to only piecewise linearly separable data. Therefore, in this paper, we extend the hard-margin model into soft-margin one by introducing penalty functions for the slack margin variables, and derive a single-objective second-order cone programming (SOCP) problem to solve it. Furthermore, through numerical experiments we verify the classification performance of the hard and soft-margin MMSVMs for benchmark problems.
Keywords: multiobjective optimization, second-order cone programming., multiclass classification, Support vector machine.
Mathematics Subject Classification: Primary: 62H30, 68T10; Secondary: 90C2.
Citation: Keiji Tatsumi, Masashi Akao, Ryo Kawachi, Tetsuzo Tanino. Performance evaluation of multiobjective multiclass support vector machines maximizing geometric margins. Numerical Algebra, Control & Optimization, 2011, 1 (1) : 151-169. doi: 10.3934/naco.2011.1.151
References:
[1]

S. Abe, "Support Vector Machines for Pattern Classification," Springer-Verlag, New York, 2005. Google Scholar

[2]

F. Alizadeh and D. Goldfarb, Second-order cone programming, Mathematical Programming, Ser. B, 95 (2003), 3-51.  Google Scholar

[3]

L. Bottou, C. Cortes, J. Denker, H. Drucker, I. Guyon, L. Jackel, Y. LeCun, U. Muller, E. Sackinger, P. Simard and V. Vapnik, Comparison of classifier methods: A case study in handwriting digit recognition, in "Proc. Int. Conf. Pattern Recognition,'' (1994), 77-87. Google Scholar

[4]

E. J. Bredensteiner and K. P. Bennett, Multicategory classification by support vector machines, Computational Optimization and Applications, 12 (1999), 53-79. doi: 10.1023/A:1008663629662.  Google Scholar

[5]

M. Ehrgott, "Multicriteria Optimization,'' 2nd edition, Springer-Verlag, Berlin, 2005.  Google Scholar

[6]

Y. Guermeur, Combining discriminant models with new multiclass SVMs, Neuro COLT2 Technical Report Series, 2000. Google Scholar

[7]

U. Kressel, Pairwise classification and support vector machines, in "Advances in kernel methods - Support vector learning'' (eds. B. Schölkopf, C. Burges, and A. J. Smola), MIT Press, Cambridge, (1999), 255-268. Google Scholar

[8]

C. W. Hsh and C. J. Lin, A comparison of methods for multiclass support vector machines, IEEE Trans. Neural Networks, 13 (2) (2002), 181-201. Google Scholar

[9]

H. D. Mittelmann, An independent benchmarking of SDP and SOCP solvers, Mathematical Programming, Ser. B, 95 (2003), 407-430.  Google Scholar

[10]

K. R. Müller, S. Mika, G. Rätsch, K. Tsuda and B. Shölkopf, An introduction to kernel-based learning algorithms, IEEE Trans. Neural Networks, 12 (2), (2001), 181-201. doi: 10.1109/72.914517.  Google Scholar

[11]

A. Passerini, M. Pontil and P. Frasconi, New results on error correcting output codes of kernel machines, IEEE Trans. Neural Networks, 14 (1), (2004), 45-54. doi: 10.1109/TNN.2003.820841.  Google Scholar

[12]

J. C. Platt, N. Cristianini and J. Shawe-Taylor, Large margin DAG's for multiclass classification, in "Advances in Neural Information Processing Systems,'' Cambridge, MA: MIT Press, 12 (2000), 547-553. Google Scholar

[13]

K. Tatsumi, K. Hayashida, H. Higashi and T. Tanino, Multi-objective multiclass support vector machine for pattern recognition, in "Proc. SICE Annual Conference 2007,'' 1095-1098. doi: 10.1109/SICE.2007.4421147.  Google Scholar

[14]

K. Tatsumi, R. Kawachi, K. Hayashida and T. Tanino, Multiobjective multiclass support vector machines maximizing geometric margins, Pacific Journal of Optimization, 6 (1), (2000), 115-140.  Google Scholar

[15]

J. S. Taylor and N. Cristianini, "Kernel Methods for Pattern Analysis,'' Cambridge University Press, 2004. Google Scholar

[16]

, UCI benchmark repository of artificial and real data sets, University of California Irvine,, Available from: , ().   Google Scholar

[17]

J. Weston and C. Watkins, Multi-class support vector machines, Technical report CSD-TR-98-04, Univ. London, Royal Holloway, (1998). Google Scholar

[18]

V. N. Vapnik, "Statistical Learning Theory,'' A Wiley-Interscience Publication, 1998.  Google Scholar

show all references

References:
[1]

S. Abe, "Support Vector Machines for Pattern Classification," Springer-Verlag, New York, 2005. Google Scholar

[2]

F. Alizadeh and D. Goldfarb, Second-order cone programming, Mathematical Programming, Ser. B, 95 (2003), 3-51.  Google Scholar

[3]

L. Bottou, C. Cortes, J. Denker, H. Drucker, I. Guyon, L. Jackel, Y. LeCun, U. Muller, E. Sackinger, P. Simard and V. Vapnik, Comparison of classifier methods: A case study in handwriting digit recognition, in "Proc. Int. Conf. Pattern Recognition,'' (1994), 77-87. Google Scholar

[4]

E. J. Bredensteiner and K. P. Bennett, Multicategory classification by support vector machines, Computational Optimization and Applications, 12 (1999), 53-79. doi: 10.1023/A:1008663629662.  Google Scholar

[5]

M. Ehrgott, "Multicriteria Optimization,'' 2nd edition, Springer-Verlag, Berlin, 2005.  Google Scholar

[6]

Y. Guermeur, Combining discriminant models with new multiclass SVMs, Neuro COLT2 Technical Report Series, 2000. Google Scholar

[7]

U. Kressel, Pairwise classification and support vector machines, in "Advances in kernel methods - Support vector learning'' (eds. B. Schölkopf, C. Burges, and A. J. Smola), MIT Press, Cambridge, (1999), 255-268. Google Scholar

[8]

C. W. Hsh and C. J. Lin, A comparison of methods for multiclass support vector machines, IEEE Trans. Neural Networks, 13 (2) (2002), 181-201. Google Scholar

[9]

H. D. Mittelmann, An independent benchmarking of SDP and SOCP solvers, Mathematical Programming, Ser. B, 95 (2003), 407-430.  Google Scholar

[10]

K. R. Müller, S. Mika, G. Rätsch, K. Tsuda and B. Shölkopf, An introduction to kernel-based learning algorithms, IEEE Trans. Neural Networks, 12 (2), (2001), 181-201. doi: 10.1109/72.914517.  Google Scholar

[11]

A. Passerini, M. Pontil and P. Frasconi, New results on error correcting output codes of kernel machines, IEEE Trans. Neural Networks, 14 (1), (2004), 45-54. doi: 10.1109/TNN.2003.820841.  Google Scholar

[12]

J. C. Platt, N. Cristianini and J. Shawe-Taylor, Large margin DAG's for multiclass classification, in "Advances in Neural Information Processing Systems,'' Cambridge, MA: MIT Press, 12 (2000), 547-553. Google Scholar

[13]

K. Tatsumi, K. Hayashida, H. Higashi and T. Tanino, Multi-objective multiclass support vector machine for pattern recognition, in "Proc. SICE Annual Conference 2007,'' 1095-1098. doi: 10.1109/SICE.2007.4421147.  Google Scholar

[14]

K. Tatsumi, R. Kawachi, K. Hayashida and T. Tanino, Multiobjective multiclass support vector machines maximizing geometric margins, Pacific Journal of Optimization, 6 (1), (2000), 115-140.  Google Scholar

[15]

J. S. Taylor and N. Cristianini, "Kernel Methods for Pattern Analysis,'' Cambridge University Press, 2004. Google Scholar

[16]

, UCI benchmark repository of artificial and real data sets, University of California Irvine,, Available from: , ().   Google Scholar

[17]

J. Weston and C. Watkins, Multi-class support vector machines, Technical report CSD-TR-98-04, Univ. London, Royal Holloway, (1998). Google Scholar

[18]

V. N. Vapnik, "Statistical Learning Theory,'' A Wiley-Interscience Publication, 1998.  Google Scholar

[1]

Yi Zhang, Yong Jiang, Liwei Zhang, Jiangzhong Zhang. A perturbation approach for an inverse linear second-order cone programming. Journal of Industrial & Management Optimization, 2013, 9 (1) : 171-189. doi: 10.3934/jimo.2013.9.171
[2]

Yubo Yuan, Weiguo Fan, Dongmei Pu. Spline function smooth support vector machine for classification. Journal of Industrial & Management Optimization, 2007, 3 (3) : 529-542. doi: 10.3934/jimo.2007.3.529
[3]

Xiaoni Chi, Zhongping Wan, Zijun Hao. Second order sufficient conditions for a class of bilevel programs with lower level second-order cone programming problem. Journal of Industrial & Management Optimization, 2015, 11 (4) : 1111-1125. doi: 10.3934/jimo.2015.11.1111
[4]

Ye Tian, Shu-Cherng Fang, Zhibin Deng, Wenxun Xing. Computable representation of the cone of nonnegative quadratic forms over a general second-order cone and its application to completely positive programming. Journal of Industrial & Management Optimization, 2013, 9 (3) : 703-721. doi: 10.3934/jimo.2013.9.703
[5]

Lin Zhu, Xinzhen Zhang. Semidefinite relaxation method for polynomial optimization with second-order cone complementarity constraints. Journal of Industrial & Management Optimization, 2021  doi: 10.3934/jimo.2021030
[6]

Liwei Zhang, Jihong Zhang, Yule Zhang. Second-order optimality conditions for cone constrained multi-objective optimization. Journal of Industrial & Management Optimization, 2018, 14 (3) : 1041-1054. doi: 10.3934/jimo.2017089
[7]

Xinmin Yang. On second order symmetric duality in nondifferentiable multiobjective programming. Journal of Industrial & Management Optimization, 2009, 5 (4) : 697-703. doi: 10.3934/jimo.2009.5.697
[8]

Liping Tang, Xinmin Yang, Ying Gao. Higher-order symmetric duality for multiobjective programming with cone constraints. Journal of Industrial & Management Optimization, 2020, 16 (4) : 1873-1884. doi: 10.3934/jimo.2019033
[9]

Anurag Jayswala, Tadeusz Antczakb, Shalini Jha. Second order modified objective function method for twice differentiable vector optimization problems over cone constraints. Numerical Algebra, Control & Optimization, 2019, 9 (2) : 133-145. doi: 10.3934/naco.2019010
[10]

Qilin Wang, Shengji Li, Kok Lay Teo. Continuity of second-order adjacent derivatives for weak perturbation maps in vector optimization. Numerical Algebra, Control & Optimization, 2011, 1 (3) : 417-433. doi: 10.3934/naco.2011.1.417
[11]

Shiyun Wang, Yong-Jin Liu, Yong Jiang. A majorized penalty approach to inverse linear second order cone programming problems. Journal of Industrial & Management Optimization, 2014, 10 (3) : 965-976. doi: 10.3934/jimo.2014.10.965
[12]

Ning Lu, Ying Liu. Application of support vector machine model in wind power prediction based on particle swarm optimization. Discrete & Continuous Dynamical Systems - S, 2015, 8 (6) : 1267-1276. doi: 10.3934/dcdss.2015.8.1267
[13]

Yubo Yuan. Canonical duality solution for alternating support vector machine. Journal of Industrial & Management Optimization, 2012, 8 (3) : 611-621. doi: 10.3934/jimo.2012.8.611
[14]

Xi-De Zhu, Li-Ping Pang, Gui-Hua Lin. Two approaches for solving mathematical programs with second-order cone complementarity constraints. Journal of Industrial & Management Optimization, 2015, 11 (3) : 951-968. doi: 10.3934/jimo.2015.11.951
[15]

Yanhong Yuan, Hongwei Zhang, Liwei Zhang. A smoothing Newton method for generalized Nash equilibrium problems with second-order cone constraints. Numerical Algebra, Control & Optimization, 2012, 2 (1) : 1-18. doi: 10.3934/naco.2012.2.1
[16]

Li Chu, Bo Wang, Jie Zhang, Hong-Wei Zhang. Convergence analysis of a smoothing SAA method for a stochastic mathematical program with second-order cone complementarity constraints. Journal of Industrial & Management Optimization, 2021, 17 (4) : 1863-1886. doi: 10.3934/jimo.2020050
[17]

Xin-He Miao, Kai Yao, Ching-Yu Yang, Jein-Shan Chen. Levenberg-Marquardt method for absolute value equation associated with second-order cone. Numerical Algebra, Control & Optimization, 2022, 12 (1) : 47-61. doi: 10.3934/naco.2021050
[18]

Narges Torabi Golsefid, Maziar Salahi. Second order cone programming formulation of the fixed cost allocation in DEA based on Nash bargaining game. Numerical Algebra, Control & Optimization, 2021  doi: 10.3934/naco.2021032
[19]

Lijun Zhang, Chaudry Masood Khalique. Classification and bifurcation of a class of second-order ODEs and its application to nonlinear PDEs. Discrete & Continuous Dynamical Systems - S, 2018, 11 (4) : 759-772. doi: 10.3934/dcdss.2018048
[20]

Qingsong Duan, Mengwei Xu, Liwei Zhang, Sainan Zhang. Hadamard directional differentiability of the optimal value of a linear second-order conic programming problem. Journal of Industrial & Management Optimization, 2021, 17 (6) : 3085-3098. doi: 10.3934/jimo.2020108

 Impact Factor: 

Tools

Metrics

  • PDF downloads (67)
  • HTML views (0)
  • Cited by (2)

Other articles
by authors

[Back to Top]

Copyright © 2022 American Institute of Mathematical Sciences | Privacy Policy