Solving higher order nonlinear ordinary differential equations with least squares support vector machines

Yanfei Lu; Qingfei Yin; Hongyi Li; Hongli Sun; Yunlei Yang; Muzhou Hou

doi:10.3934/jimo.2019012

Article Contents

2020, Volume 16, Issue 3: 1481-1502. Doi: 10.3934/jimo.2019012

This issue Previous Article Optimal reinsurance-investment problem with dependent risks based on Legendre transform Next Article A class of accelerated conjugate-gradient-like methods based on a modified secant equation

Solving higher order nonlinear ordinary differential equations with least squares support vector machines

School of Mathematics and Statistics, Central South University, Changsha 410083, Hunan, China

^* Corresponding author: Muzhou Hou
^* Corresponding author: Muzhou Hou

Received: March 31, 2018

Revised: September 30, 2018

Early access: March 2019

Published: March 2020

The second author is supported by the National Social Science Foundation of China (13BTJ010)

Abstract Full Text(HTML) Figure(6) / Table(14) Related Papers Cited by

Abstract

In this paper, a numerical method based on least squares support vector machines has been developed to solve the initial and boundary value problems of higher order nonlinear ordinary differential equations. The numerical experiments have been performed on some nonlinear ordinary differential equations to validate the accuracy and reliability of our proposed LS–SVM model. Compared with the exact solution, the results obtained by our proposed LS–SVM model can achieve a very high accuracy. The proposed LS–SVM model could be a good tool for solving higher order nonlinear ordinary differential equations.

Keywords:

Nonlinear ordinary differential equations,
initial value problems,
boundary value problems,
least squares support vector machines,
approximate solutions.

Mathematics Subject Classification: Primary: 34A45; Secondary: 68W25, 68T99.

Citation:

Full Text(HTML)

Figure 1. Second-order nonlinear BVP

Download: Full-size image PowerPoint slide

Figure 2. The logarithmic relation between σ and MSE(Example 1)

Download: Full-size image PowerPoint slide

Figure 3. Second-order nonlinear IVP

Download: Full-size image PowerPoint slide

Figure 4. M-order nonlinear IVP

Download: Full-size image PowerPoint slide

Figure 5. Second order nonlinear BVP (Example 4)

Download: Full-size image PowerPoint slide

Figure 6. M-order nonlinear BVP(Example 5)

Download: Full-size image PowerPoint slide

Table 1. Comparison between the exact solution and the approximate solution obtained by LS-SVM model for the training points (Example 1)

Training data	Exact solution	LS-SVM	Training data	Exact solution	LS-SVM
$ 1.000 $	1.000000000	1.000000000	$ 1.550 $	1.215686275	1.215686275
$ 1.050 $	1.024390244	1.024390233	$ 1.600 $	1.230769231	1.230769232
$ 1.100 $	1.047619048	1.047619039	$ 1.650 $	1.245283019	1.245283019
$ 1.150 $	1.069767442	1.069767431	$ 1.700 $	1.259259259	1.259259258
$ 1.200 $	1.090909091	1.090909081	$ 1.750 $	1.272727273	1.272727273
$ 1.250 $	1.111111111	1.111111105	$ 1.800 $	1.285714286	1.285714289
$ 1.300 $	1.130434783	1.130434779	$ 1.850 $	1.298245614	1.298245619
$ 1.350 $	1.148936170	1.148936165	$ 1.900 $	1.310344828	1.310344831
$ 1.400 $	1.166666667	1.166666660	$ 1.950 $	1.322033898	1.322033904
$ 1.450 $	1.183673469	1.183673463	$ 2.000 $	1.333333333	1.333333333
$ 1.500 $	1.200000000	1.199999997	$

| Show Table

DownLoad: CSV

Table 2. Exact and LS-SVM results for the testing points(Example 1)

Testing data	Exact solution	LS-SVM	Testing data	Exact solution	LS-SVM
$ 1.0250 $	1.012345679	1.012345667	$ 1.5250 $	1.207920792	1.207920791
$ 1.0750 $	1.036144578	1.036144569	$ 1.5750 $	1.223300971	1.223300972
$ 1.1250 $	1.058823529	1.058823519	$ 1.6250 $	1.238095238	1.238095239
$ 1.1750 $	1.080459770	1.080459759	$ 1.6750 $	1.252336449	1.252336448
$ 1.2250 $	1.101123596	1.101123588	$ 1.7250 $	1.266055046	1.266055045
$ 1.2750 $	1.120879121	1.120879117	$ 1.7750 $	1.279279279	1.279279281
$ 1.3250 $	1.139784946	1.139784942	$ 1.8250 $	1.292035398	1.292035403
$ 1.3750 $	1.157894737	1.157894731	$ 1.8750 $	1.304347826	1.304347830
$ 1.4250 $	1.175257732	1.175257725	$ 1.9250 $	1.316239317	1.316239320
$ 1.4750 $	1.191919192	1.191919187	$ 1.9750 $	1.327731092	1.327731099

| Show Table

DownLoad: CSV

Table 3. Comparison of the MSE with the differential number of the training points(Example 1)

N	$ \sigma $	$ MSE_{test} $
$ 11 $	1.50	$ 2.2982\times10^{-14} $
$ 21 $	0.90	$ 3.2973\times10^{-17} $
$ 41 $	0.60	$ 1.3527\times10^{-19} $

| Show Table

DownLoad: CSV

Table 4. Comparison between the exact solution and the approximate solution obtained by LS-SVM model for the training points(Example 2)

Training data	Exact solution	LS-SVM	Training data	Exact solution	LS-SVM
$ 0.0000 $	1.000000000	1.000000000	$ 0.2750 $	0.7843137255	0.783960261
$ 0.0250 $	0.975609756	0.975507638	$ 0.3000 $	0.7692307692	0.768994225
$ 0.0500 $	0.952380952	0.952038212	$ 0.3250 $	0.7547169811	0.754586586
$ 0.0750 $	0.930232558	0.929751041	$ 0.3500 $	0.7407407407	0.740709174
$ 0.1000 $	0.909090909	0.908583285	$ 0.3750 $	0.7272727273	0.727354567
$ 0.1250 $	0.888888889	0.888395226	$ 0.4000 $	0.7142857143	0.714516534
$ 0.1500 $	0.869565217	0.869067566	$ 0.4250 $	0.7017543860	0.702168828
$ 0.1750 $	0.851063830	0.850537044	$ 0.4500 $	0.6896551724	0.690259179
$ 0.2000 $	0.833333333	0.832783164	$ 0.4750 $	0.6779661017	0.678730163
$ 0.2250 $	0.816326531	0.815793928	$ 0.5000 $	0.6666666667	0.667567043
$ 0.2500 $	0.800000000	0.799538249	$

| Show Table

DownLoad: CSV

Table 5. Exact and LS-SVM results for the testing points(Example 2)

Testing data	Exact solution	LS-SVM	Testing data	Exact solution	LS-SVM
$ 0.0125 $	0.9876543210	0.9876489423	$ 0.2625 $	0.7920792079	0.7916686867
$ 0.0375 $	0.9638554217	0.9636280130	$ 0.2875 $	0.7766990291	0.7764046131
$ 0.0625 $	0.9411764706	0.9407474275	$ 0.3125 $	0.7619047619	0.7617230005
$ 0.0875 $	0.9195402299	0.9190356064	$ 0.3375 $	0.7476635514	0.7475823709
$ 0.1125 $	0.8988764045	0.8983754394	$ 0.3625 $	0.7339449541	0.7339666547
$ 0.1375 $	0.8791208791	0.8786291359	$ 0.3875 $	0.7207207207	0.7208719671
$ 0.1625 $	0.8602150538	0.8597045711	$ 0.4125 $	0.7079646018	0.7082841451
$ 0.1875 $	0.8421052632	0.8415635623	$ 0.4375 $	0.6956521739	0.6961631623
$ 0.2125 $	0.8247422680	0.8241942736	$ 0.4625 $	0.6837606838	0.6844497170
$ 0.2375 $	0.8080808089	0.8075774037	$ 0.4875 $	0.6722689076	0.6731004361

| Show Table

DownLoad: CSV

Table 6. Comparison of the MSE with the differential number of the training points(Example 2)

N	$ \sigma $	$ MSE_{test} $
$ 11 $	0.1625	$ 4.1363\times10^{-7} $
$ 21 $	0.2235	$ 1.9693\times10^{-7} $
$ 41 $	0.4085	$ 1.8597\times10^{-7} $

| Show Table

DownLoad: CSV

Table 7. Comparison between the exact solution and the approximate solution obtained by LS-SVM model for the training points(Example 3)

Example 3	Exact solution	LS-SVM
$ 0.000 $	0.00000000000	0.00000000000
$ 0.050 $	0.04997916927	0.04997915236
$ 0.100 $	0.09983341665	0.09983310498
$ 0.150 $	0.14943813247	0.14943658985
$ 0.200 $	0.19866933080	0.19866476297
$ 0.250 $	0.24740395925	0.24739375649
$ 0.300 $	0.29552020666	0.29550123081
$ 0.350 $	0.34289780746	0.34286692221
$ 0.400 $	0.38941834231	0.38937318159
$ 0.450 $	0.43496553411	0.43490549976
$ 0.500 $	0.47942553860	0.47935301511

| Show Table

DownLoad: CSV

Table 8. Exact and LS-SVM results for the testing points(Example 3)

Example 3	Exact solution	LS-SVM
$ 0.0250 $	0.02499739591	0.02499739536
$ 0.0750 $	0.07492970727	0.07492961138
$ 0.1250 $	0.12467473339	0.12467397503
$ 0.1750 $	0.17410813759	0.17410536148
$ 0.2250 $	0.22310636213	0.22309934567
$ 0.2750 $	0.27154693696	0.27153275506
$ 0.3250 $	0.31930878586	0.31928421995
$ 0.3750 $	0.36627252909	0.36623471736
$ 0.4250 $	0.41232078174	0.41226810347
$ 0.4750 $	0.45733844718	0.45727163071

| Show Table

DownLoad: CSV

Table 9. Comparison of the MSE with the differential number of the training points(Example 3)

N	$ \sigma $	$ MSE_{test} $
$ 11 $	2.1200	$ 9.5313\times10^{-10} $
$ 21 $	4.3750	$ 1.2437\times10^{-10} $
$ 41 $	4.4595	$ 1.1478\times10^{-13} $

| Show Table

DownLoad: CSV

Table 10. Comparison between bvp4c and the approximate solution obtained by LS-SVM model for the training points(Example 4)

Example 4	Bvp4c	LS-SVM
$ 0.000 $	0.00000000000	0.00000000000
$ 0.100 $	0.08208961709	0.07269777630
$ 0.200 $	0.16500909145	0.15151008989
$ 0.300 $	0.24963062540	0.23642312027
$ 0.400 $	0.33691524509	0.32742078522
$ 0.500 $	0.42796806735	0.42448474566
$ 0.600 $	0.52410852807	0.52759441177
$ 0.700 $	0.62696468677	0.63672695027
$ 0.800 $	0.73860596417	0.75185729280
$ 0.900 $	0.86173796851	0.87295814556
$ 1.000 $	1.00000000000	1.00000190735

| Show Table

DownLoad: CSV

Table 11. Bvp4c and LS-SVM results for the testing points(Example 4)

Example 4	Bvp4c	LS-SVM
$ 0.050 $	0.04099337478	0.03558379567
$ 0.150 $	0.12339244872	0.11134042665
$ 0.250 $	0.20704941350	0.19320496766
$ 0.350 $	0.29287463208	0.28116246679
$ 0.450 $	0.38189348425	0.37519571270
$ 0.550 $	0.47530819486	0.47528524009
$ 0.650 $	0.57457869449	0.58140933646
$ 0.750 $	0.68153387549	0.69354404985
$ 0.850 $	0.79853157226	0.81166319791
$ 0.950 $	0.92869807782	0.93573837804

| Show Table

DownLoad: CSV

Table 12. Comparison between the exact solution and the approximate solution obtained by LS-SVM model for the training points(Example 5)

Example 5	Exact solution	LS-SVM
$ 0.000 $	0.00000000000	0.00000000000
$ 0.100 $	0.00072900000	0.00072904178
$ 0.200 $	0.00409600000	0.00409607260
$ 0.300 $	0.00926100000	0.01382410696
$ 0.400 $	0.01382400000	0.01390350516
$ 0.500 $	0.01562500000	0.01562510374
$ 0.600 $	0.01382400000	0.01382410574
$ 0.700 $	0.00926100000	0.00926110345
$ 0.800 $	0.00409600000	0.00409607300
$ 0.900 $	0.00072900000	0.00072904190
$ 1.000 $	0.00000000000	0.00000000000

| Show Table

DownLoad: CSV

Table 13. Exact and LS-SVM results for the testing points(Example 5)

Example 5	Exact solution	LS-SVM
$ 0.050 $	0.000107171875	0.000107186927
$ 0.150 $	0.002072671875	0.002072728701
$ 0.250 $	0.006591796875	0.006591885737
$ 0.350 $	0.011774546875	0.011774653640
$ 0.450 $	0.015160921875	0.015161024270
$ 0.550 $	0.015160921875	0.015161026227
$ 0.650 $	0.011774546875	0.011774654858
$ 0.750 $	0.006591796875	0.006591885679
$ 0.850 $	0.002072671875	0.002072728375
$ 0.950 $	0.000107171875	0.000107189695

| Show Table

DownLoad: CSV

Table 14. Comparison of the MSE with the differential number of the training points(Example 5)

N	$ \sigma $	$ MSE_{test} $
$ 11 $	1.8000	$ 6.7181\times10^{-15} $
$ 21 $	0.7000	$ 6.9489\times10^{-18} $
$ 41 $	0.2001	$ 1.6352\times10^{-19} $

| Show Table

DownLoad: CSV

Related Papers

Cited by

References

[1]	D. O. Awoyemi, A p–stable linear multistep method for solving general third order ordinary differential equations, International Journal of Computer Mathematics, 80 (2003), 987-993. doi: 10.1080/0020716031000079572.
[2]	S. Chakraverty and S. Mall, Regression based weight generation algorithm in neural network for solution of initial and boundary value problems, Neural Computing and Applications, 25 (2014), 585-594. doi: 10.1007/s00521-013-1526-4.
[3]	E. H. Doha, A. H. Bhrawy and R. M. Hafez, On shifted Jacobi spectral method for high–order multi–point boundary value problems, Communications in Nonlinear Science and Numerical Simulation, 17 (2012), 3802-3810. doi: 10.1016/j.cnsns.2012.02.027.
[4]	N. M. Duy, H. See and T. T. Cong, A spectral collocation technique based on integrated chebyshev polynomials for biharmonic problems in irregular domains, Applied Mathematical Modelling, 33 (2009), 284-299. doi: 10.1016/j.apm.2007.11.002.
[5]	T. Hofmann, B. Schölkopf and A. J. Smola, Kernel methods in machine learning, Annals of Statistics, 36 (2008), 1171-1220. doi: 10.1214/009053607000000677.
[6]	G. B. Huang, Q. Y. Zhu and C. K. Siew, Extreme learning machine: Theory and applications, Neurocomputing, 70 (2006), 489-501. doi: 10.1016/j.neucom.2005.12.126.
[7]	K. Hussain, F. Ismail, N. Senu and F. Rabiei, Fourth–order improved Runge–Kutta method for directly solving special third–order ordinary differential equations, Iranian Journal of Science and Technology Transaction A–Science, 41 (2017), 429-437. doi: 10.1007/s40995-017-0258-1.
[8]	K. Hussain, F. Ismail and N. Senu, Solving directly special fourth–order ordinary differential equations using Runge–Kutta type method, Journal of Computational and Applied Mathematics, 306 (2016), 179-199. doi: 10.1016/j.cam.2016.04.002.
[9]	S. Islam, I. Aziz and B. Šarler, The numerical solution of second–order boundary value problems by collocation method with the Haar wavelets, Mathematical and Computer Modelling, 52 (2010), 1577-1590. doi: 10.1016/j.mcm.2010.06.023.
[10]	D. R. Kincaid and E. W. Cheney, Numerical Analysis: Mathematics of Scientific Computing, 3^nd edition, Brooks/Cole, Pacific Grove, CA, 1991.
[11]	J. Kierzenka and L. F. Shampine, A BVP solver that controls residual and error, Journal of Numerical Analysis, Industrial and Applied Mathematics, 3 (2008), 27-41.
[12]	M. Lakestani and M. Dehgan, The solution of a second–order nonlinear differential equation with Neumann boundary conditions using semi–orthogonal B–spline wavelets, International Journal of Computer Mathematics, 83 (2006), 685-694. doi: 10.1080/00207160601025656.
[13]	Z. A. Majid and M. Suleiman, Direct integration method implicit variable steps method for solving higher order systems of ordinary differential equations directly, Sains Malaysiana, 35 (2006), 63-68.
[14]	Z. A. Majid, N. A. Azmi, M. Suleiman and Z. B. Ibrahaim, Solving directly general third order ordinary differential equations using two–point four step block method, Sains Malaysiana, 41 (2012), 623-632.
[15]	A. Malek and R. S. Beidokhti, Numerical solution for high order differential equations using a hybrid neural network–optimization method, Applied Mathematics and Computation, 183 (2006), 260-271. doi: 10.1016/j.amc.2006.05.068.
[16]	S. Mall and S. Chakraverty, Application of Legendre neural network for solving ordinary differential equations, Applied Soft Computing, 43 (2016), 347-356.
[17]	S. Mall and S. Chakraverty, Chebyshev neural network based model for solving Lane–Emden type equations, Applied Mathematics and Computation, 247 (2014), 100-114. doi: 10.1016/j.amc.2014.08.085.
[18]	S. Mehrkanoon, A direct variable step block multistep method for solving general third–order ODEs, Numerical Algorithms, 57 (2011), 53-66. doi: 10.1007/s11075-010-9413-x.
[19]	S. Mehrkanoon, T. Falck and J. A. K. Suykens, Approximate solutions to ordinary differential equations using least squares support vector machines, IEEE Transactions on Neural Networks and Learning Systems, 23 (2012), 1356-1367. doi: 10.1109/TNNLS.2012.2202126.
[20]	S. Mehrkanoon and J. A. K. Suykens, Learning solutions to partial differential equations using LS–SVM, Neurocomputing, 159 (2015), 105-116. doi: 10.1016/j.neucom.2015.02.013.
[21]	S. Mehrkanoon and J. A. K. Suykens, LS–SVM approximate solution to linear time varying descriptor systems, Automatica, 48 (2012), 2502-2511. doi: 10.1016/j.automatica.2012.06.095.
[22]	J. Mercer, Functions of positive and negative type and their connection with the theory of integral equations, Philosophical Transactions of the Royal Society of London, 209 (1909), 415-446.
[23]	R. Noberg, Differential equations for moments of present values in life insurance, Mathematics and Economics, 17 (1995), 171-180. doi: 10.1016/0167-6687(95)00019-O.
[24]	K. Parand and M. Hemami, Numerical study of astrophysics equations by meshless collocation method based on compactly supported radial basis function, International Journal of Applied and Computational Mathematics, 3 (2017), 1053-1075. doi: 10.1007/s40819-016-0161-z.
[25]	H. J. Ricardo, A Modern Introduction to Differential Equations, CRC Press, Boca Raton, FL, 2010.
[26]	P. P. See, Z. A. Majid and M. Suleiman, Three–step block method for solving nonlinear boundary value problems, Abstract and Applied Analysis, 2014 (2014), Art. ID 379829, 8 pp. doi: 10.1155/2014/379829.
[27]	P. K. Srivastava, M. Kumar and R. N. Mohapatra, Quintic nonpolynomial spline method for the solution of a second–order boundary value problem with engineering applications, Computers and Mathematics with Applications, 62 (2011), 1707-1714. doi: 10.1016/j.camwa.2011.06.012.
[28]	H. Sun, M. Hou, Y. Yang, T. Zhang, F. Weng and F. Han, Solving partial differential equation based on Bernstein neural network and extreme learning machine algorithm, Neural Processing Letters, (2018), 1–20. doi: 10.1007/s11063-018-9911-8.
[29]	J. A. K. Suykens and J. Vandewalle, Least squares support vector machine classifiers, Neural Processing Letters, 9 (1999) 293–300.
[30]	I. A. Tirmizi and E. H. Twizell, Higher–order finite difference methods for nonlinear second–order two point boundary value problems, Applied Mathematics Letter, 15 (2002), 897-902. doi: 10.1016/S0893-9659(02)00060-5.
[31]	V. N. Vapnik, The Nature of Statistical Learning Theory, 1^nd edition, Springer–Verlag, New York, 1995. doi: 10.1007/978-1-4757-2440-0.
[32]	Q. Wang, K. Wang and S. Chen, Least squares approximation method for the solution of Volterra–Fredholm integral equations, Journal of Computational and Applied Mathematics, 272 (2014), 141-147. doi: 10.1016/j.cam.2014.05.010.
[33]	H. S. Yazdi, M. Pakdaman and H. Modaghegh, Unsupervised kernel least mean square algorithm for solving ordinary differential equations, Neurocomputing, 74 (2011), 2062-2071.
[34]	G. Zhang, S. Wang and Y. Wang, LS–SVM approximate solution for affine nonlinear systems with partially unknown functions, Journal of Industrial and Management Optimization, 10 (2014), 621-636. doi: 10.3934/jimo.2014.10.621.