Article Contents
Article Contents

# Solving higher order nonlinear ordinary differential equations with least squares support vector machines

• * Corresponding author: Muzhou Hou

The second author is supported by the National Social Science Foundation of China (13BTJ010)

• In this paper, a numerical method based on least squares support vector machines has been developed to solve the initial and boundary value problems of higher order nonlinear ordinary differential equations. The numerical experiments have been performed on some nonlinear ordinary differential equations to validate the accuracy and reliability of our proposed LS–SVM model. Compared with the exact solution, the results obtained by our proposed LS–SVM model can achieve a very high accuracy. The proposed LS–SVM model could be a good tool for solving higher order nonlinear ordinary differential equations.

Mathematics Subject Classification: Primary: 34A45; Secondary: 68W25, 68T99.

 Citation:

• Figure 1.  Second-order nonlinear BVP

Figure 2.  The logarithmic relation between σ and MSE(Example 1)

Figure 3.  Second-order nonlinear IVP

Figure 4.  M-order nonlinear IVP

Figure 5.  Second order nonlinear BVP (Example 4)

Figure 6.  M-order nonlinear BVP(Example 5)

Table 1.  Comparison between the exact solution and the approximate solution obtained by LS-SVM model for the training points (Example 1)

 Training data Exact solution LS-SVM Training data Exact solution LS-SVM $1.000$ 1.000000000 1.000000000 $1.550$ 1.215686275 1.215686275 $1.050$ 1.024390244 1.024390233 $1.600$ 1.230769231 1.230769232 $1.100$ 1.047619048 1.047619039 $1.650$ 1.245283019 1.245283019 $1.150$ 1.069767442 1.069767431 $1.700$ 1.259259259 1.259259258 $1.200$ 1.090909091 1.090909081 $1.750$ 1.272727273 1.272727273 $1.250$ 1.111111111 1.111111105 $1.800$ 1.285714286 1.285714289 $1.300$ 1.130434783 1.130434779 $1.850$ 1.298245614 1.298245619 $1.350$ 1.148936170 1.148936165 $1.900$ 1.310344828 1.310344831 $1.400$ 1.166666667 1.166666660 $1.950$ 1.322033898 1.322033904 $1.450$ 1.183673469 1.183673463 $2.000$ 1.333333333 1.333333333 $1.500$ 1.200000000 1.199999997 $Table 2. Exact and LS-SVM results for the testing points(Example 1)  Testing data Exact solution LS-SVM Testing data Exact solution LS-SVM$ 1.0250 $1.012345679 1.012345667$ 1.5250 $1.207920792 1.207920791$ 1.0750 $1.036144578 1.036144569$ 1.5750 $1.223300971 1.223300972$ 1.1250 $1.058823529 1.058823519$ 1.6250 $1.238095238 1.238095239$ 1.1750 $1.080459770 1.080459759$ 1.6750 $1.252336449 1.252336448$ 1.2250 $1.101123596 1.101123588$ 1.7250 $1.266055046 1.266055045$ 1.2750 $1.120879121 1.120879117$ 1.7750 $1.279279279 1.279279281$ 1.3250 $1.139784946 1.139784942$ 1.8250 $1.292035398 1.292035403$ 1.3750 $1.157894737 1.157894731$ 1.8750 $1.304347826 1.304347830$ 1.4250 $1.175257732 1.175257725$ 1.9250 $1.316239317 1.316239320$ 1.4750 $1.191919192 1.191919187$ 1.9750 $1.327731092 1.327731099 Table 3. Comparison of the MSE with the differential number of the training points(Example 1)  N$ \sigma  MSE_{test}  11 $1.50$ 2.2982\times10^{-14}  21 $0.90$ 3.2973\times10^{-17}  41 $0.60$ 1.3527\times10^{-19} $Table 4. Comparison between the exact solution and the approximate solution obtained by LS-SVM model for the training points(Example 2)  Training data Exact solution LS-SVM Training data Exact solution LS-SVM$ 0.0000 $1.000000000 1.000000000$ 0.2750 $0.7843137255 0.783960261$ 0.0250 $0.975609756 0.975507638$ 0.3000 $0.7692307692 0.768994225$ 0.0500 $0.952380952 0.952038212$ 0.3250 $0.7547169811 0.754586586$ 0.0750 $0.930232558 0.929751041$ 0.3500 $0.7407407407 0.740709174$ 0.1000 $0.909090909 0.908583285$ 0.3750 $0.7272727273 0.727354567$ 0.1250 $0.888888889 0.888395226$ 0.4000 $0.7142857143 0.714516534$ 0.1500 $0.869565217 0.869067566$ 0.4250 $0.7017543860 0.702168828$ 0.1750 $0.851063830 0.850537044$ 0.4500 $0.6896551724 0.690259179$ 0.2000 $0.833333333 0.832783164$ 0.4750 $0.6779661017 0.678730163$ 0.2250 $0.816326531 0.815793928$ 0.5000 $0.6666666667 0.667567043$ 0.2500 $0.800000000 0.799538249$

Table 5.  Exact and LS-SVM results for the testing points(Example 2)

 Testing data Exact solution LS-SVM Testing data Exact solution LS-SVM $0.0125$ 0.9876543210 0.9876489423 $0.2625$ 0.7920792079 0.7916686867 $0.0375$ 0.9638554217 0.9636280130 $0.2875$ 0.7766990291 0.7764046131 $0.0625$ 0.9411764706 0.9407474275 $0.3125$ 0.7619047619 0.7617230005 $0.0875$ 0.9195402299 0.9190356064 $0.3375$ 0.7476635514 0.7475823709 $0.1125$ 0.8988764045 0.8983754394 $0.3625$ 0.7339449541 0.7339666547 $0.1375$ 0.8791208791 0.8786291359 $0.3875$ 0.7207207207 0.7208719671 $0.1625$ 0.8602150538 0.8597045711 $0.4125$ 0.7079646018 0.7082841451 $0.1875$ 0.8421052632 0.8415635623 $0.4375$ 0.6956521739 0.6961631623 $0.2125$ 0.8247422680 0.8241942736 $0.4625$ 0.6837606838 0.6844497170 $0.2375$ 0.8080808089 0.8075774037 $0.4875$ 0.6722689076 0.6731004361

Table 6.  Comparison of the MSE with the differential number of the training points(Example 2)

 N $\sigma$ $MSE_{test}$ $11$ 0.1625 $4.1363\times10^{-7}$ $21$ 0.2235 $1.9693\times10^{-7}$ $41$ 0.4085 $1.8597\times10^{-7}$

Table 7.  Comparison between the exact solution and the approximate solution obtained by LS-SVM model for the training points(Example 3)

 Example 3 Exact solution LS-SVM $0.000$ 0.00000000000 0.00000000000 $0.050$ 0.04997916927 0.04997915236 $0.100$ 0.09983341665 0.09983310498 $0.150$ 0.14943813247 0.14943658985 $0.200$ 0.19866933080 0.19866476297 $0.250$ 0.24740395925 0.24739375649 $0.300$ 0.29552020666 0.29550123081 $0.350$ 0.34289780746 0.34286692221 $0.400$ 0.38941834231 0.38937318159 $0.450$ 0.43496553411 0.43490549976 $0.500$ 0.47942553860 0.47935301511

Table 8.  Exact and LS-SVM results for the testing points(Example 3)

 Example 3 Exact solution LS-SVM $0.0250$ 0.02499739591 0.02499739536 $0.0750$ 0.07492970727 0.07492961138 $0.1250$ 0.12467473339 0.12467397503 $0.1750$ 0.17410813759 0.17410536148 $0.2250$ 0.22310636213 0.22309934567 $0.2750$ 0.27154693696 0.27153275506 $0.3250$ 0.31930878586 0.31928421995 $0.3750$ 0.36627252909 0.36623471736 $0.4250$ 0.41232078174 0.41226810347 $0.4750$ 0.45733844718 0.45727163071

Table 9.  Comparison of the MSE with the differential number of the training points(Example 3)

 N $\sigma$ $MSE_{test}$ $11$ 2.1200 $9.5313\times10^{-10}$ $21$ 4.3750 $1.2437\times10^{-10}$ $41$ 4.4595 $1.1478\times10^{-13}$

Table 10.  Comparison between bvp4c and the approximate solution obtained by LS-SVM model for the training points(Example 4)

 Example 4 Bvp4c LS-SVM $0.000$ 0.00000000000 0.00000000000 $0.100$ 0.08208961709 0.07269777630 $0.200$ 0.16500909145 0.15151008989 $0.300$ 0.24963062540 0.23642312027 $0.400$ 0.33691524509 0.32742078522 $0.500$ 0.42796806735 0.42448474566 $0.600$ 0.52410852807 0.52759441177 $0.700$ 0.62696468677 0.63672695027 $0.800$ 0.73860596417 0.75185729280 $0.900$ 0.86173796851 0.87295814556 $1.000$ 1.00000000000 1.00000190735

Table 11.  Bvp4c and LS-SVM results for the testing points(Example 4)

 Example 4 Bvp4c LS-SVM $0.050$ 0.04099337478 0.03558379567 $0.150$ 0.12339244872 0.11134042665 $0.250$ 0.20704941350 0.19320496766 $0.350$ 0.29287463208 0.28116246679 $0.450$ 0.38189348425 0.37519571270 $0.550$ 0.47530819486 0.47528524009 $0.650$ 0.57457869449 0.58140933646 $0.750$ 0.68153387549 0.69354404985 $0.850$ 0.79853157226 0.81166319791 $0.950$ 0.92869807782 0.93573837804

Table 12.  Comparison between the exact solution and the approximate solution obtained by LS-SVM model for the training points(Example 5)

 Example 5 Exact solution LS-SVM $0.000$ 0.00000000000 0.00000000000 $0.100$ 0.00072900000 0.00072904178 $0.200$ 0.00409600000 0.00409607260 $0.300$ 0.00926100000 0.01382410696 $0.400$ 0.01382400000 0.01390350516 $0.500$ 0.01562500000 0.01562510374 $0.600$ 0.01382400000 0.01382410574 $0.700$ 0.00926100000 0.00926110345 $0.800$ 0.00409600000 0.00409607300 $0.900$ 0.00072900000 0.00072904190 $1.000$ 0.00000000000 0.00000000000

Table 13.  Exact and LS-SVM results for the testing points(Example 5)

 Example 5 Exact solution LS-SVM $0.050$ 0.000107171875 0.000107186927 $0.150$ 0.002072671875 0.002072728701 $0.250$ 0.006591796875 0.006591885737 $0.350$ 0.011774546875 0.011774653640 $0.450$ 0.015160921875 0.015161024270 $0.550$ 0.015160921875 0.015161026227 $0.650$ 0.011774546875 0.011774654858 $0.750$ 0.006591796875 0.006591885679 $0.850$ 0.002072671875 0.002072728375 $0.950$ 0.000107171875 0.000107189695

Table 14.  Comparison of the MSE with the differential number of the training points(Example 5)

 N $\sigma$ $MSE_{test}$ $11$ 1.8000 $6.7181\times10^{-15}$ $21$ 0.7000 $6.9489\times10^{-18}$ $41$ 0.2001 $1.6352\times10^{-19}$
•  [1] D. O. Awoyemi, A p–stable linear multistep method for solving general third order ordinary differential equations, International Journal of Computer Mathematics, 80 (2003), 987-993.  doi: 10.1080/0020716031000079572. [2] S. Chakraverty and S. Mall, Regression based weight generation algorithm in neural network for solution of initial and boundary value problems, Neural Computing and Applications, 25 (2014), 585-594.  doi: 10.1007/s00521-013-1526-4. [3] E. H. Doha, A. H. Bhrawy and R. M. Hafez, On shifted Jacobi spectral method for high–order multi–point boundary value problems, Communications in Nonlinear Science and Numerical Simulation, 17 (2012), 3802-3810.  doi: 10.1016/j.cnsns.2012.02.027. [4] N. M. Duy, H. See and T. T. Cong, A spectral collocation technique based on integrated chebyshev polynomials for biharmonic problems in irregular domains, Applied Mathematical Modelling, 33 (2009), 284-299.  doi: 10.1016/j.apm.2007.11.002. [5] T. Hofmann, B. Schölkopf and A. J. Smola, Kernel methods in machine learning, Annals of Statistics, 36 (2008), 1171-1220.  doi: 10.1214/009053607000000677. [6] G. B. Huang, Q. Y. Zhu and C. K. Siew, Extreme learning machine: Theory and applications, Neurocomputing, 70 (2006), 489-501.  doi: 10.1016/j.neucom.2005.12.126. [7] K. Hussain, F. Ismail, N. Senu and F. Rabiei, Fourth–order improved Runge–Kutta method for directly solving special third–order ordinary differential equations, Iranian Journal of Science and Technology Transaction A–Science, 41 (2017), 429-437.  doi: 10.1007/s40995-017-0258-1. [8] K. Hussain, F. Ismail and N. Senu, Solving directly special fourth–order ordinary differential equations using Runge–Kutta type method, Journal of Computational and Applied Mathematics, 306 (2016), 179-199.  doi: 10.1016/j.cam.2016.04.002. [9] S. Islam, I. Aziz and B. Šarler, The numerical solution of second–order boundary value problems by collocation method with the Haar wavelets, Mathematical and Computer Modelling, 52 (2010), 1577-1590.  doi: 10.1016/j.mcm.2010.06.023. [10] D. R. Kincaid and E. W. Cheney, Numerical Analysis: Mathematics of Scientific Computing, 3nd edition, Brooks/Cole, Pacific Grove, CA, 1991. [11] J. Kierzenka and L. F. Shampine, A BVP solver that controls residual and error, Journal of Numerical Analysis, Industrial and Applied Mathematics, 3 (2008), 27-41. [12] M. Lakestani and M. Dehgan, The solution of a second–order nonlinear differential equation with Neumann boundary conditions using semi–orthogonal B–spline wavelets, International Journal of Computer Mathematics, 83 (2006), 685-694.  doi: 10.1080/00207160601025656. [13] Z. A. Majid and M. Suleiman, Direct integration method implicit variable steps method for solving higher order systems of ordinary differential equations directly, Sains Malaysiana, 35 (2006), 63-68. [14] Z. A. Majid, N. A. Azmi, M. Suleiman and Z. B. Ibrahaim, Solving directly general third order ordinary differential equations using two–point four step block method, Sains Malaysiana, 41 (2012), 623-632. [15] A. Malek and R. S. Beidokhti, Numerical solution for high order differential equations using a hybrid neural network–optimization method, Applied Mathematics and Computation, 183 (2006), 260-271.  doi: 10.1016/j.amc.2006.05.068. [16] S. Mall and S. Chakraverty, Application of Legendre neural network for solving ordinary differential equations, Applied Soft Computing, 43 (2016), 347-356. [17] S. Mall and S. Chakraverty, Chebyshev neural network based model for solving Lane–Emden type equations, Applied Mathematics and Computation, 247 (2014), 100-114.  doi: 10.1016/j.amc.2014.08.085. [18] S. Mehrkanoon, A direct variable step block multistep method for solving general third–order ODEs, Numerical Algorithms, 57 (2011), 53-66.  doi: 10.1007/s11075-010-9413-x. [19] S. Mehrkanoon, T. Falck and J. A. K. Suykens, Approximate solutions to ordinary differential equations using least squares support vector machines, IEEE Transactions on Neural Networks and Learning Systems, 23 (2012), 1356-1367.  doi: 10.1109/TNNLS.2012.2202126. [20] S. Mehrkanoon and J. A. K. Suykens, Learning solutions to partial differential equations using LS–SVM, Neurocomputing, 159 (2015), 105-116.  doi: 10.1016/j.neucom.2015.02.013. [21] S. Mehrkanoon and J. A. K. Suykens, LS–SVM approximate solution to linear time varying descriptor systems, Automatica, 48 (2012), 2502-2511.  doi: 10.1016/j.automatica.2012.06.095. [22] J. Mercer, Functions of positive and negative type and their connection with the theory of integral equations, Philosophical Transactions of the Royal Society of London, 209 (1909), 415-446. [23] R. Noberg, Differential equations for moments of present values in life insurance, Mathematics and Economics, 17 (1995), 171-180.  doi: 10.1016/0167-6687(95)00019-O. [24] K. Parand and M. Hemami, Numerical study of astrophysics equations by meshless collocation method based on compactly supported radial basis function, International Journal of Applied and Computational Mathematics, 3 (2017), 1053-1075.  doi: 10.1007/s40819-016-0161-z. [25] H. J. Ricardo,  A Modern Introduction to Differential Equations, CRC Press, Boca Raton, FL, 2010. [26] P. P. See, Z. A. Majid and M. Suleiman, Three–step block method for solving nonlinear boundary value problems, Abstract and Applied Analysis, 2014 (2014), Art. ID 379829, 8 pp. doi: 10.1155/2014/379829. [27] P. K. Srivastava, M. Kumar and R. N. Mohapatra, Quintic nonpolynomial spline method for the solution of a second–order boundary value problem with engineering applications, Computers and Mathematics with Applications, 62 (2011), 1707-1714.  doi: 10.1016/j.camwa.2011.06.012. [28] H. Sun, M. Hou, Y. Yang, T. Zhang, F. Weng and F. Han, Solving partial differential equation based on Bernstein neural network and extreme learning machine algorithm, Neural Processing Letters, (2018), 1–20. doi: 10.1007/s11063-018-9911-8. [29] J. A. K. Suykens and J. Vandewalle, Least squares support vector machine classifiers, Neural Processing Letters, 9 (1999) 293–300. [30] I. A. Tirmizi and E. H. Twizell, Higher–order finite difference methods for nonlinear second–order two point boundary value problems, Applied Mathematics Letter, 15 (2002), 897-902.  doi: 10.1016/S0893-9659(02)00060-5. [31] V. N. Vapnik, The Nature of Statistical Learning Theory, 1nd edition, Springer–Verlag, New York, 1995. doi: 10.1007/978-1-4757-2440-0. [32] Q. Wang, K. Wang and S. Chen, Least squares approximation method for the solution of Volterra–Fredholm integral equations, Journal of Computational and Applied Mathematics, 272 (2014), 141-147.  doi: 10.1016/j.cam.2014.05.010. [33] H. S. Yazdi, M. Pakdaman and H. Modaghegh, Unsupervised kernel least mean square algorithm for solving ordinary differential equations, Neurocomputing, 74 (2011), 2062-2071. [34] G. Zhang, S. Wang and Y. Wang, LS–SVM approximate solution for affine nonlinear systems with partially unknown functions, Journal of Industrial and Management Optimization, 10 (2014), 621-636.  doi: 10.3934/jimo.2014.10.621.

Figures(6)

Tables(14)