\`x^2+y_1+z_12^34\`
Advanced Search
Article Contents
Article Contents

Hybrid binary dragonfly enhanced particle swarm optimization algorithm for solving feature selection problems

  • * Corresponding author: Mohamed A. Tawhid

    * Corresponding author: Mohamed A. Tawhid 
We are grateful to the anonymous 4 reviewers for constructive feedback and insightful suggestions which greatly improved this article. This research was supported partially by Mitacs Canada. The research of the 1st author is supported in part by the Natural Sciences and Engineering Research Council of Canada (NSERC).
Abstract Full Text(HTML) Figure(2) / Table(9) Related Papers Cited by
  • In this paper, we present a new hybrid binary version of dragonfly and enhanced particle swarm optimization algorithm in order to solve feature selection problems. The proposed algorithm is called Hybrid Binary Dragonfly Enhanced Particle Swarm Optimization Algorithm(HBDESPO). In the proposed HBDESPO algorithm, we combine the dragonfly algorithm with its ability to encourage diverse solutions with its formation of static swarms and the enhanced version of the particle swarm optimization exploiting the data with its ability to converge to the best global solution in the search space. In order to investigate the general performance of the proposed HBDESPO algorithm, the proposed algorithm is compared with the original optimizers and other optimizers that have been used for feature selection in the past. Further, we use a set of assessment indicators to evaluate and compare the different optimizers over 20 standard data sets obtained from the UCI repository. Results prove the ability of the proposed HBDESPO algorithm to search the feature space for optimal feature combinations.

    Mathematics Subject Classification: Primary: 68T20, 68T01, 68U20; Secondary: 62P10, 62P30.

    Citation:

    \begin{equation} \\ \end{equation}
  • 加载中
  • Figure 1.  The Comparison of performance the HBDEPSO algorithm with other optimizers through main objectives of feature selection. The values are averaged over all the datasets

    Figure 2.  The Comparison of performance the HBDEPSO algorithm with other optimizers through few assessment indicators. The values are averaged over all the datasets

    Table 1.  Datasets

    Dataset# of Attributes# of Instances
    Zoo16101
    WineEW13178
    IonosphereEW34351
    WaveformEW405000
    BreastEW30569
    Breastcancer9699
    Congress16435
    Exactly131000
    Exactly2131000
    HeartEW13270
    KrvskpEW363196
    M-of-n131000
    SonarEW60208
    SpectEW60208
    Tic-tac-toe9958
    Lymphography18148
    Dermatology34366
    Echocardiogram12132
    hepatitis19155
    LungCancer5632
     | Show Table
    DownLoad: CSV

    Table 2.  Parameter setting

    Parameter Value
    No of iterations($max_{iter}$)70
    No of search agents($n$)5
    Dimension($D$)No. of features in the data
    Search domain[0 1]
    No of runs($M$)10
    $w_{max}$0.9
    $w_{min}$0.4
    $Deltax_{max}$6
    $c_1$2
    $c_2$2
    $v_{max}$6
    $\beta$ in fitness function0.01
    $\alpha$ in fitness function0.99
     | Show Table
    DownLoad: CSV

    Table 3.  Mean fitness function obtained from the different algorithms

    Dataset HBDESPO BDA EPSO BGA BBA BGWO2 HBEPSOD
    Zoo0.0400.067 0.0310.1240.0940.1190.082
    Wine EW 0.0360.0500.0420.0650.1280.0920.041
    IonosphereEW 0.1100.1300.1370.1430.1460.1720.115
    WaveformEW0.1790.183 0.1750.1860.1930.1850.175
    BreastEW 0.0400.0570.0500.1060.0700.0800.044
    Breastcancer 0.0230.0320.0320.0360.0350.0420.030
    Congress 0.0280.0420.0330.0590.0530.0730.036
    Exactly 0.1030.1780.1040.2690.3030.3160.139
    Exactly2 0.2240.2400.2340.2430.2430.2630.241
    HeartEW 0.1250.1530.1530.2500.2400.2680.128
    KrvskpEW0.0440.0410.0430.0890.1080.080 0.039
    M-of-n0.0250.048 0.0240.1080.1670.1540.084
    SonarEW 0.1580.1940.1920.2620.2770.2900.179
    SpectEW0.148 0.1330.1600.1680.1670.2050.142
    Tic-tac-toe 0.2220.2230.2220.2410.2700.2620.227
    Lymphography 0.3810.3920.4120.4660.4870.5310.426
    Dermatology 0.0160.0170.0160.0310.0810.0990.017
    Echocardiogram 0.0510.0580.0830.0720.1120.2000.074
    Hepatitis0.118 0.1010.1230.1520.1750.1920.115
    LungCancer 0.2190.2550.2200.3180.4270.4550.291
    Average 0.1140.1310.1230.1690.1890.2040.131
     | Show Table
    DownLoad: CSV

    Table 4.  Best fitness function obtained from the different algorithms

    Dataset HBDESPO BDA EPSO BGA BBA BGWO2 HBEPSOD
    Zoo 0.0000.0000.0010.0320.0050.0350.004
    Wine EW 0.0020.0030.0190.0350.0210.0030.019
    IonosphereEW 0.0710.1080.1130.1140.0790.0890.096
    WaveformEW0.1710.1810.1650.1740.1760.167 0.162
    BreastEW 0.0250.0550.0270.0600.0450.0560.034
    Breastcancer0.0140.0240.0180.0290.0240.0270.014
    Congress 0.0160.0190.0220.0380.0290.0450.022
    Exactly 0.0040.0400.0250.0580.2700.2980.025
    Exactly2 0.2110.2350.2190.2160.2120.2410.220
    HeartEW0.091 0.0820.1040.1470.1680.1470.082
    KrvskpEW0.0410.0340.0330.0410.0600.059 0.029
    M-of-n 0.0040.0040.0040.0670.1130.1280.004
    SonarEW 0.1180.1560.1180.2200.2050.2340.134
    SpectEW0.115 0.0930.1250.1250.1270.1610.115
    Tic-tac-toe0.2130.206 0.1850.2170.2360.2420.196
    Lymphography 0.2860.3440.3070.3880.4270.4500.349
    Dermatology 0.0030.0030.0040.0120.0290.0460.004
    Echocardiogram 0.0030.0250.0470.0450.0490.0930.047
    Hepatitis 0.0580.0580.0800.0780.1170.0970.061
    LungCancer0.093 0.0030.0580.0930.1840.280.094
    Average 0.0770.0840.0840.1100.1290.1450.086
     | Show Table
    DownLoad: CSV

    Table 5.  Worst fitness function obtained from the different algorithms

    Dataset HBDESPO BDA EPSO BGA BBA BGWO2 HBEPSOD
    Zoo0.1210.208 0.0890.2080.2080.2080.206
    Wine EW 0.0690.1190.0700.1220.2730.1570.070
    IonosphereEW 0.1460.1550.1710.1890.1910.3090.147
    WaveformEW 0.1840.1920.1860.1970.2150.1950.186
    BreastEW 0.0490.0650.0810.3150.1030.1150.054
    Breastcancer 0.0310.0390.0410.0490.0490.0520.038
    Congress 0.0430.0630.0490.0850.0920.0890.049
    Exactly 0.2130.3080.2510.3490.3260.3420.294
    Exactly2 0.2380.2630.2480.2680.2760.2860.265
    HeartEW 0.1680.2010.2890.3220.3340.3570.168
    KrvskpEW 0.0470.0520.0540.1770.1910.1010.063
    M-of-n 0.0490.1360.0730.1570.2320.1700.461
    SonarEW 0.1910.2340.2190.3060.3910.3490.262
    SpectEW 0.1700.1700.2040.2050.2160.2380.192
    Tic-tac-toe 0.2360.2390.2440.2750.3130.2980.243
    Lymphography 0.4680.4910.4690.5880.5690.5810.549
    Dermatology 0.0290.0530.0290.0610.2900.2220.030
    Echocardiogram 0.0700.0920.160.1140.230.8400.115
    Hepatitis0.230 0.1380.1740.2120.2340.2530.175
    LungCancer0.542 0.4540.5430.7230.8130.5450.722
    Average 0.1650.1840.1820.2460.2770.2850.214
     | Show Table
    DownLoad: CSV

    Table 6.  Standard deviation of the fitness function obtained from the different algorithms

    Dataset HBDESPO BDA EPSO BGA BBA BGWO2 HBEPSOD
    Zoo0.0520.075 0.0330.0660.0700.0670.056
    Wine EW0.0190.0300.0180.0260.0800.057 0.017
    IonosphereEW0.0220.0180.0160.0250.0400.057 0.013
    WaveformEW 0.0030.0060.0080.0080.01230.0070.006
    BreastEW 0.0060.0070.0190.7550.0170.0180.006
    Breastcancer 0.0050.0050.0070.0070.0090.0080.009
    Congress 0.0070.0160.0080.0130.0190.0150.008
    Exactly0.0710.1190.0820.0780.020 0.0160.117
    Exactly2 0.0090.0150.0090.0190.0170.0180.015
    HeartEW 0.0250.0360.0550.0620.0640.0690.025
    KrvskpEW 0.0020.0070.0070.0510.0440.0120.010
    M-of-n 0.0180.0510.0220.0320.0360.0190.136
    SonarEW 0.0270.0330.0290.0300.0590.0430.037
    SpectEW 0.0160.0220.0270.0290.0280.0240.029
    Tic-tac-toe 0.0070.0120.0200.0210.0250.0170.014
    Lymphography0.0520.0490.0480.0620.047 0.0440.055
    Dermatology 0.0070.0140.0080.0140.0750.0500.008
    Echocardiogram 0.0240.0260.0300.0240.0550.2280.025
    Hepatitis0.050 0.0250.0280.0380.0430.0520.030
    LungCancer 0.0920.1510.1800.2330.1940.0930.183
    Average 0.0260.0360.0330.0800.0480.0460.040
     | Show Table
    DownLoad: CSV

    Table 7.  Average performance of the selected features by different algorithms

    Dataset HBDESPO BDA EPSO BGA BBA BGWO2 HBEPSOD
    Zoo0.8440.7880.791 0.8630.7990.8510.852
    Wine EW0.916 0.9230.8810.8860.7260.8960.888
    IonosphereEW 0.8350.7990.8290.8280.8170.8240.810
    WaveformEW 0.8230.8070.8090.8060.7790.8190.806
    BreastEW 0.9490.9440.9310.8920.8420.9080.926
    Breastcancer 0.9600.9560.9560.9570.9570.9570.958
    Congress 0.9450.9310.9430.9150.8930.9280.935
    Exactly 0.8950.7980.8840.6870.6470.6800.846
    Exactly2 0.7460.7390.7380.7340.7110.7320.736
    HeartEW 0.8150.810.7760.7110.6480.7020.811
    KrvskpEW 0.9590.9540.9580.9060.7720.9170.958
    M-of-n 0.9780.9490.9750.8920.7190.8430.957
    SonarEW 0.7050.6580.6820.6940.6780.6820.682
    SpectEW0.7620.7520.7570.7500.755 0.7770.747
    Tic-tac-toe 0.7480.7450.7400.7340.6470.7130.737
    Lymphography0.4060.4170.3540.416 0.4220.3790.411
    Dermatology 0.9580.9400.9520.950.8020.9080.945
    Echocardiogram0.8750.893 0.9060.8520.8610.8770.863
    Hepatitis 0.8190.7880.8130.7980.7880.7880.803
    LungCancer 0.4810.4270.3900.4090.3430.3450.345
    Average 0.8210.8010.8030.7840.7300.7760.801
     | Show Table
    DownLoad: CSV

    Table 8.  Average selected feature ratio by different algorithms

    Dataset HBDESPO BDA EPSO BGA BBA BGWO2 HBEPSOD
    Zoo 0.2930.3310.3560.4120.5120.4730.4
    Wine EW 0.2840.3380.40.3150.5380.5160.338
    IonosphereEW 0.3670.3970.3880.4020.5260.5410.397
    WaveformEW 0.6330.6660.7090.6760.63410.752
    BreastEW 0.2410.2830.2410.2900.4800.4700.3
    Breastcancer 0.4110.4220.5110.5660.5110.6440.544
    Congress 0.3060.3370.3250.4120.4930.5750.318
    Exactly 0.4690.5070.5070.5610.5380.5760.523
    Exactly2 0.3920.3920.4920.40.5460.80.415
    HeartEW 0.3910.4070.4070.4150.4920.4300.4
    KrvskpEW0.486 0.4750.5020.5300.5130.6330.516
    M-of-n0.5150.5300.4760.576 0.4460.9230.515
    SonarEW0.44 0.4130.4630.420.5210.5330.475
    SpectEW 0.4130.4540.4630.4250.4810.5290.440
    Tic-tac-toe0.5550.5550.666 0.5110.5770.8660.533
    Lymphography 0.390.4380.4160.40.4610.5350.45
    Dermatology0.5 0.4110.5110.4790.4940.5440.5
    Echocardiogram 0.2250.2330.2660.2830.5080.4830.25
    Hepatitis0.2730.2730.321 0.2310.5150.4310.294
    LungCancer 0.350.3530.4230.3800.4980.5260.357
    Average 0.3970.4110.4420.4340.5140.6010.436
     | Show Table
    DownLoad: CSV

    Table 9.  Average Fischer index of the selected features by different algorithms

    Dataset HBDESPO BDA EPSO BGA BBA BGWO2 HBEPSOD
    Zoo 161105143156112130140
    Wine EW24.13374.67648.48540.5211784.7 20939.741.169
    IonosphereEW3.6023.9863.8704.7694.154 5.0424.056
    WaveformEW2.3552.3142.3142.1652.029 3.4562.454
    BreastEW 7.2E+132.5E+113.3E+131.4E+135.7E+126.9E+133.1E+11
    Breastcancer 1.1900.7481.0700.9230.9421.1050.884
    Congress 48.58431.79713.99613.04511.00318.31722.088
    Exactly 0.3910.2590.1310.3780.3500.2820.144
    Exactly2 0.3950.2400.2670.2870.2000.2370.227
    HeartEW3.7883.4243.357140.64161.62 430.072.197
    KrvskpEW 1396.5544.21940.241023.2639.911187.5913.89
    M-of-n 1.7911.7111.7351.7861.6521.3731.693
    SonarEW6.4E+67.3E+68.2E+65.5E+68.2E+6 1.2E+79.5E+6
    SpectEW 0.0080.0060.0050.0040.0060.0060.006
    Tic-tac-toe 0.1680.0900.1610.1190.1360.1170.134
    Lymphography 9.773.139.182.434.412.733.51
    Dermatology 400269343148210174207
    Echocardiogram158.28579.06137662931 13093953037985.60
    Hepatitis5.9633.49151.8014.420132.03 5303784.211
    LungCancer 42.97331.14840.20329.40530.81033.61522.220
    Average 3.6E+121.3E+101.6E+126.9E+112.8E+113.4E+111.5E+10
     | Show Table
    DownLoad: CSV
  • [1] D. K. Agrafiotis and W. Cedeno, Feature selection for structure-activity correlation using binary particle swarms, Journal of Medicinal Chemistry, 45 (2002), 1098-1107. 
    [2] H. Banati and M. Bajaj, Fire fly based feature selection approach, IJCSI International Journal of Computer Science Issues, 8 (2011).
    [3] D. Bell and H. Wang, A formalism for relevance and its application in feature subset selection, Mach. Learn., 41 (2000), 175-195. 
    [4] B. XueM. ZhangW. Browne and X. Yao, A survey on evolutionary computation approaches to feature selection, IEEE Transaction on Evolutionary Computation, 20 (2016), 606-626.  doi: 10.1109/TEVC.2015.2504420.
    [5] G. Chandrashekar and F. Sahin, A survey on feature selection methods, Electrical and Microelectronic Engineering, Rochester Institute of Technology, Rochester, NY 14623, USA, 2013.
    [6] B. ChiziL. Rokach and O. Maimon, A survey of feature selection techniques, Encyclopedia of Data Warehousing and Mining, seconded, IGI Global, (2009), 1888-1895. 
    [7] L. Y. ChuangH. W. ChangC. J. Tu and C. H. Yang, Improved binary PSO for feature selection using gene expression data, Comput.Biol.Chem., 32 (2008), 29-38. 
    [8] G. Coath and S. K. Halgamuge, A comparison of constraint-handling methods for the application of particle swarm optimization to constrained nonlinear optimization problems, Proceedings of IEEE Congress on Evolutionary Computation 2003 (CEC 2003), Canbella, Australia, (2003), 2419–2425.
    [9] C. A. Coello CoelloE. H. Luna and A. H. Aguirre, Use of particle swarm optimization to design combinational logic circuits, International Conference on Evolvable Systems, (2003), 398-409.  doi: 10.1007/3-540-36553-2_36.
    [10] C. Cotta, A study of hybridisation techniques and their application to the design of evolutionary algorithms, AI Communications, 11 (1998), 223-224. 
    [11] R. C. Eberhart and J. Kennedy, A new optimizer using particle swarm theory. Proceedings of the Sixth International Symposium on Micromachine and Human Science, Nagoya, Japan, (1995), 39–43.
    [12] E. EmaryH. M. ZawbaaC. Grosan and A. E. Hassanien, Binary grey wolf optimization approaches for feature selection, Neurocomputing, Elsevier, 172 (2016), 371-381. 
    [13] A. Frank and A. Asuncion, UCI Machine Learning Repository, 2010.
    [14] J. HuangY. Cai and X. Xu, A hybrid genetic algorithm for feature selection wrapper based on mutual information, Pattern Recognition Letters archive, 28 (2007), 1825-1844.  doi: 10.1016/j.patrec.2007.05.011.
    [15] J. Kennedy, R. C. Eberhart and Y. Shi, Swarm Intelligence, Morgan Kaufmann, SanMateo, CA, 2001.
    [16] S. Khalid, A survey of feature selection and feature extraction techniques in machine learning, Science and Information Conference (SAI), 2014.
    [17] R. A. Krohling, H. Knidel and Y. Shi, Solving numerical equations of hydraulic problems using particle swarm optimization, Proceedings of the IEEE Congress on Evolutionary Computation (CEC 2002), Honolulu, Hawaii USA, 2002.
    [18] S. Mirjalili and A. Lewis, S-shaped versus V-shaped transfer functions for binary Particle Swarm Optimization, Swarm and Evolutionary Computation, 9 (2012), 1-14. 
    [19] S. Mirjalili, Dragonfly algorithm: A new meta-heuristic optimization technique for solving single-objective, discrete, and multi-objective problems, Neural Computing and Applications, 27 (2016), 1053-1073. 
    [20] R. Y. M. NakamuraL. A. M. PereiraK. A. CostaD. RodriguesJ. P. Papa and X.-S. Yang, Binary bat algorithm for feature selection, Conference on Graphics, Patterns and Images, (2012), 291-297. 
    [21] Q. Gu, Z. Li and J. Han, Generalized Fisher Score for Feature Selection, In Proc. of the 27th Conference on Uncertainty in Artificial Intelligence (UAI), Barcelona, Spain, 2011.
    [22] E. G. Talbi, A taxonomy of hybrid metaheuristics, Journal of Heuristics, 8 (2002), 541-565. 
    [23] D. Wolpert and W. Macready, No free lunch theorems for optimization, IEEE Transactions on Evolutionary Computation, 1 (1997), 67-72. 
  • 加载中

Figures(2)

Tables(9)

SHARE

Article Metrics

HTML views(4776) PDF downloads(487) Cited by(0)

Access History

Other Articles By Authors

Catalog

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return