• Previous Article
    An integrated approach for the operations of distribution and lateral transshipment for seasonal products - A case study in household product industry
  • JIMO Home
  • This Issue
  • Next Article
    Multi-objective aggregate production planning decisions using two-phase fuzzy goal programming method
April  2011, 7(2): 385-400. doi: 10.3934/jimo.2011.7.385

Nonlinear dynamical system modeling via recurrent neural networks and a weighted state space search algorithm

1. 

Department of Applied Mathematics, The Hong Kong Polytechnic University, Kowloon, Hong Kong, China

2. 

Department of Mathematics, Cleveland State University, Cleveland, OH 44115, United States

Received  October 2009 Revised  January 2011 Published  April 2011

Given a task of tracking a trajectory, a recurrent neural network may be considered as a black-box nonlinear regression model for tracking unknown dynamic systems. An error function is used to measure the difference between the system outputs and the desired trajectory that formulates a nonlinear least square problem with dynamical constraints. With the dynamical constraints, classical gradient type methods are difficult and time consuming due to the involving of the computation of the partial derivatives along the trajectory. We develop an alternative learning algorithm, namely the weighted state space search algorithm, which searches the neighborhood of the target trajectory in the state space instead of the parameter space. Since there is no computation of partial derivatives involved, our algorithm is simple and fast. We demonstrate our approach by modeling the short-term foreign exchange rates. The empirical results show that the weighted state space search method is very promising and effective in solving least square problems with dynamical constraints. Numerical costs between the gradient method and our the proposed method are provided.
Citation: Leong-Kwan Li, Sally Shao, K. F. Cedric Yiu. Nonlinear dynamical system modeling via recurrent neural networks and a weighted state space search algorithm. Journal of Industrial & Management Optimization, 2011, 7 (2) : 385-400. doi: 10.3934/jimo.2011.7.385
References:
[1]

A. F. Atiya and A. G. Parlos, New results on recurrent network training: Unifying the algorithms and accelerating convergence,, IEEE Transcations on Neural Networks, 11 (2000), 697.  doi: 10.1109/72.846741.  Google Scholar

[2]

Y. Fang and T. W. S. Chow, Non-linear dynamical systems control using a new RNN temporal learning strategy,, IEEE Trans on Circuit and Systems, 52 (2005), 719.   Google Scholar

[3]

R. A. Conn, K. Scheinberg and N. L. Vicente, "Introduction to Derivative-free Optimization,", SIAM, (2009).  doi: 10.1137/1.9780898718768.  Google Scholar

[4]

J. F. G. Freitas, M. Niranjan, A. H. Gee and A. Doucet, Sequential Monte Carlo methods to train neural network models,, Neural Computation, 12 (2000), 955.  doi: 10.1162/089976600300015664.  Google Scholar

[5]

L. K. Li, Learning sunspot series dynamics by recurrent neural networks,, in, (2003), 107.  doi: 10.1142/9789812704955_0009.  Google Scholar

[6]

L. K. Li, W. K. Pang, W. T. Yu and M. D. Trout, Forecasting short-term exchange Rates: a recurrent neural network approach,, in, (2004), 195.  doi: 10.4018/9781591401766.ch010.  Google Scholar

[7]

L. K. Li and S. Shao, Dynamic properties of recurrent neural networks and its approximations,, International Journal of Pure and Applied Mathematics, 39 (2007), 545.   Google Scholar

[8]

L. K. Li and S. Shao, A neural network approach for global optimization with applications,, Neural Network World, 18 (2008), 365.   Google Scholar

[9]

L. K. Li, S. Shao and T. Zheleva, A state space search algorithm and its application to learn the short-term foreign exchange rates,, Applied Mathematical Sciences, 2 (2008), 1705.   Google Scholar

[10]

X. D. Li, J. K. L. Ho and T. W. S. Chow, Approximation of dynamical time-variant systems by continuous-time recurrent neural networks,, IEEE Trans on Circuit and Systems, 52 (2005), 656.   Google Scholar

[11]

X. B. Liang and J. Wang, A recurrent neural network for nonlinear optimization with a continuously differentiable objective function and bound constraints,, IEEE Transactions on Neural Networks, 11 (2000), 1251.  doi: 10.1109/72.883412.  Google Scholar

[12]

Z. Liu and I. Elhanany, A Fast and Scalable Recurrent Neural Network Based on Stochastic Meta Descent,, IEEE Transactions on Neural Networks, 19 (2008), 1652.  doi: 10.1109/TNN.2008.2000838.  Google Scholar

[13]

S. Wang, Q. Shao and X. Zhou, Knot-optimizing spline networks (KOSNETS) for nonparametric regression,, Journal of Industrial and Management Optimization, 4 (2008).   Google Scholar

[14]

X. Wang and E. K. Blum, Discrete-time versus continuous-time models of neural networks,, Journal of Computer and System Sciences, 45 (1992), 1.  doi: 10.1016/0022-0000(92)90038-K.  Google Scholar

[15]

R. J. Williams and D. Zipser, A learning algorithm for continually running fully recurrent neural networks,, Neural Computation, 1 (1989), 270.  doi: 10.1162/neco.1989.1.2.270.  Google Scholar

[16]

L. Xu and W. Liu, A new recurrent neural network adaptive approach for host-gate way rate control protocol within intranets using ATM ABR service,, Journal of Industrial and Management Optimization, 1 (2005), 389.   Google Scholar

[17]

J. Yao and C. L. Tan, A case study on using neural networks to perform technical forecasting of forex,, Neural Computation, 34 (2000), 79.   Google Scholar

[18]

K. F. C. Yiu, S. Wang, K. L. Teo and A. H. Tsoi, Nonlinear system modeling via knot-optimizing B-spline networks,, IEEE Transactions on Neural Networks, 12 (2001), 1013.  doi: 10.1109/72.950131.  Google Scholar

[19]

K. F. C. Yiu, Y. Liu and K. L. Teo, A hybrid descent method for global optimization,, Journal of Global Optimization, 28 (2004), 229.  doi: 10.1023/B:JOGO.0000015313.93974.b0.  Google Scholar

show all references

References:
[1]

A. F. Atiya and A. G. Parlos, New results on recurrent network training: Unifying the algorithms and accelerating convergence,, IEEE Transcations on Neural Networks, 11 (2000), 697.  doi: 10.1109/72.846741.  Google Scholar

[2]

Y. Fang and T. W. S. Chow, Non-linear dynamical systems control using a new RNN temporal learning strategy,, IEEE Trans on Circuit and Systems, 52 (2005), 719.   Google Scholar

[3]

R. A. Conn, K. Scheinberg and N. L. Vicente, "Introduction to Derivative-free Optimization,", SIAM, (2009).  doi: 10.1137/1.9780898718768.  Google Scholar

[4]

J. F. G. Freitas, M. Niranjan, A. H. Gee and A. Doucet, Sequential Monte Carlo methods to train neural network models,, Neural Computation, 12 (2000), 955.  doi: 10.1162/089976600300015664.  Google Scholar

[5]

L. K. Li, Learning sunspot series dynamics by recurrent neural networks,, in, (2003), 107.  doi: 10.1142/9789812704955_0009.  Google Scholar

[6]

L. K. Li, W. K. Pang, W. T. Yu and M. D. Trout, Forecasting short-term exchange Rates: a recurrent neural network approach,, in, (2004), 195.  doi: 10.4018/9781591401766.ch010.  Google Scholar

[7]

L. K. Li and S. Shao, Dynamic properties of recurrent neural networks and its approximations,, International Journal of Pure and Applied Mathematics, 39 (2007), 545.   Google Scholar

[8]

L. K. Li and S. Shao, A neural network approach for global optimization with applications,, Neural Network World, 18 (2008), 365.   Google Scholar

[9]

L. K. Li, S. Shao and T. Zheleva, A state space search algorithm and its application to learn the short-term foreign exchange rates,, Applied Mathematical Sciences, 2 (2008), 1705.   Google Scholar

[10]

X. D. Li, J. K. L. Ho and T. W. S. Chow, Approximation of dynamical time-variant systems by continuous-time recurrent neural networks,, IEEE Trans on Circuit and Systems, 52 (2005), 656.   Google Scholar

[11]

X. B. Liang and J. Wang, A recurrent neural network for nonlinear optimization with a continuously differentiable objective function and bound constraints,, IEEE Transactions on Neural Networks, 11 (2000), 1251.  doi: 10.1109/72.883412.  Google Scholar

[12]

Z. Liu and I. Elhanany, A Fast and Scalable Recurrent Neural Network Based on Stochastic Meta Descent,, IEEE Transactions on Neural Networks, 19 (2008), 1652.  doi: 10.1109/TNN.2008.2000838.  Google Scholar

[13]

S. Wang, Q. Shao and X. Zhou, Knot-optimizing spline networks (KOSNETS) for nonparametric regression,, Journal of Industrial and Management Optimization, 4 (2008).   Google Scholar

[14]

X. Wang and E. K. Blum, Discrete-time versus continuous-time models of neural networks,, Journal of Computer and System Sciences, 45 (1992), 1.  doi: 10.1016/0022-0000(92)90038-K.  Google Scholar

[15]

R. J. Williams and D. Zipser, A learning algorithm for continually running fully recurrent neural networks,, Neural Computation, 1 (1989), 270.  doi: 10.1162/neco.1989.1.2.270.  Google Scholar

[16]

L. Xu and W. Liu, A new recurrent neural network adaptive approach for host-gate way rate control protocol within intranets using ATM ABR service,, Journal of Industrial and Management Optimization, 1 (2005), 389.   Google Scholar

[17]

J. Yao and C. L. Tan, A case study on using neural networks to perform technical forecasting of forex,, Neural Computation, 34 (2000), 79.   Google Scholar

[18]

K. F. C. Yiu, S. Wang, K. L. Teo and A. H. Tsoi, Nonlinear system modeling via knot-optimizing B-spline networks,, IEEE Transactions on Neural Networks, 12 (2001), 1013.  doi: 10.1109/72.950131.  Google Scholar

[19]

K. F. C. Yiu, Y. Liu and K. L. Teo, A hybrid descent method for global optimization,, Journal of Global Optimization, 28 (2004), 229.  doi: 10.1023/B:JOGO.0000015313.93974.b0.  Google Scholar

[1]

Rui Hu, Yuan Yuan. Stability, bifurcation analysis in a neural network model with delay and diffusion. Conference Publications, 2009, 2009 (Special) : 367-376. doi: 10.3934/proc.2009.2009.367

[2]

Pengfei Wang, Mengyi Zhang, Huan Su. Input-to-state stability of infinite-dimensional stochastic nonlinear systems. Discrete & Continuous Dynamical Systems - B, 2021  doi: 10.3934/dcdsb.2021066

[3]

Xu Zhang, Xiang Li. Modeling and identification of dynamical system with Genetic Regulation in batch fermentation of glycerol. Numerical Algebra, Control & Optimization, 2015, 5 (4) : 393-403. doi: 10.3934/naco.2015.5.393

[4]

Yanqin Fang, Jihui Zhang. Multiplicity of solutions for the nonlinear Schrödinger-Maxwell system. Communications on Pure & Applied Analysis, 2011, 10 (4) : 1267-1279. doi: 10.3934/cpaa.2011.10.1267

[5]

Manoel J. Dos Santos, Baowei Feng, Dilberto S. Almeida Júnior, Mauro L. Santos. Global and exponential attractors for a nonlinear porous elastic system with delay term. Discrete & Continuous Dynamical Systems - B, 2021, 26 (5) : 2805-2828. doi: 10.3934/dcdsb.2020206

[6]

Marita Holtmannspötter, Arnd Rösch, Boris Vexler. A priori error estimates for the space-time finite element discretization of an optimal control problem governed by a coupled linear PDE-ODE system. Mathematical Control & Related Fields, 2021  doi: 10.3934/mcrf.2021014

[7]

Hong Yi, Chunlai Mu, Guangyu Xu, Pan Dai. A blow-up result for the chemotaxis system with nonlinear signal production and logistic source. Discrete & Continuous Dynamical Systems - B, 2021, 26 (5) : 2537-2559. doi: 10.3934/dcdsb.2020194

[8]

Kuan-Hsiang Wang. An eigenvalue problem for nonlinear Schrödinger-Poisson system with steep potential well. Communications on Pure & Applied Analysis, , () : -. doi: 10.3934/cpaa.2021030

[9]

Tobias Geiger, Daniel Wachsmuth, Gerd Wachsmuth. Optimal control of ODEs with state suprema. Mathematical Control & Related Fields, 2021  doi: 10.3934/mcrf.2021012

[10]

Khosro Sayevand, Valeyollah Moradi. A robust computational framework for analyzing fractional dynamical systems. Discrete & Continuous Dynamical Systems - S, 2021  doi: 10.3934/dcdss.2021022

[11]

Jingni Guo, Junxiang Xu, Zhenggang He, Wei Liao. Research on cascading failure modes and attack strategies of multimodal transport network. Journal of Industrial & Management Optimization, 2021  doi: 10.3934/jimo.2020159

[12]

Andrey Kovtanyuk, Alexander Chebotarev, Nikolai Botkin, Varvara Turova, Irina Sidorenko, Renée Lampe. Modeling the pressure distribution in a spatially averaged cerebral capillary network. Mathematical Control & Related Fields, 2021  doi: 10.3934/mcrf.2021016

[13]

Zengyun Wang, Jinde Cao, Zuowei Cai, Lihong Huang. Finite-time stability of impulsive differential inclusion: Applications to discontinuous impulsive neural networks. Discrete & Continuous Dynamical Systems - B, 2021, 26 (5) : 2677-2692. doi: 10.3934/dcdsb.2020200

[14]

Guirong Jiang, Qishao Lu. The dynamics of a Prey-Predator model with impulsive state feedback control. Discrete & Continuous Dynamical Systems - B, 2006, 6 (6) : 1301-1320. doi: 10.3934/dcdsb.2006.6.1301

[15]

Samir Adly, Oanh Chau, Mohamed Rochdi. Solvability of a class of thermal dynamical contact problems with subdifferential conditions. Numerical Algebra, Control & Optimization, 2012, 2 (1) : 91-104. doi: 10.3934/naco.2012.2.91

[16]

Yunfei Lv, Rong Yuan, Yuan He. Wavefronts of a stage structured model with state--dependent delay. Discrete & Continuous Dynamical Systems - A, 2015, 35 (10) : 4931-4954. doi: 10.3934/dcds.2015.35.4931

[17]

Xinyuan Liao, Caidi Zhao, Shengfan Zhou. Compact uniform attractors for dissipative non-autonomous lattice dynamical systems. Communications on Pure & Applied Analysis, 2007, 6 (4) : 1087-1111. doi: 10.3934/cpaa.2007.6.1087

[18]

Daoyuan Fang, Ting Zhang. Compressible Navier-Stokes equations with vacuum state in one dimension. Communications on Pure & Applied Analysis, 2004, 3 (4) : 675-694. doi: 10.3934/cpaa.2004.3.675

[19]

Alina Chertock, Alexander Kurganov, Mária Lukáčová-Medvi${\rm{\check{d}}}$ová, Șeyma Nur Özcan. An asymptotic preserving scheme for kinetic chemotaxis models in two space dimensions. Kinetic & Related Models, 2019, 12 (1) : 195-216. doi: 10.3934/krm.2019009

[20]

Wei Liu, Pavel Krejčí, Guoju Ye. Continuity properties of Prandtl-Ishlinskii operators in the space of regulated functions. Discrete & Continuous Dynamical Systems - B, 2017, 22 (10) : 3783-3795. doi: 10.3934/dcdsb.2017190

2019 Impact Factor: 1.366

Metrics

  • PDF downloads (103)
  • HTML views (0)
  • Cited by (1)

Other articles
by authors

[Back to Top]