American Institute of Mathematical Sciences

Advanced
September  2017, 12(3): 417-459. doi: 10.3934/nhm.2017019

Optimal control of infinite-dimensional piecewise deterministic Markov processes and application to the control of neuronal dynamics via Optogenetics

Vincent Renault 1, , Michèle Thieullen 1, and  Emmanuel Trélat 2,

1. 

Sorbonne Universités, UPMC Univ Paris 06, CNRS UMR 7599, Laboratoire de Probabilités et Modèles Aléatoires, F-75005, Paris, France
2. 

Sorbonne Universités, UPMC Univ Paris 06, CNRS UMR 7598, Laboratoire Jacques-Louis Lions, F-75005, Paris, France

Received  July 2016 Revised  May 2017 Published  September 2017

In this paper we define an infinite-dimensional controlled piecewise deterministic Markov process (PDMP) and we study an optimal control problem with finite time horizon and unbounded cost. This process is a coupling between a continuous time Markov Chain and a set of semilinear parabolic partial differential equations, both processes depending on the control. We apply dynamic programming to the embedded Markov decision process to obtain existence of optimal relaxed controls and we give some sufficient conditions ensuring the existence of an optimal ordinary control. This study, which constitutes an extension of controlled PDMPs to infinite dimension, is motivated by the control that provides Optogenetics on neuron models such as the Hodgkin-Huxley model. We define an infinite-dimensional controlled Hodgkin-Huxley model as an infinite-dimensional controlled piecewise deterministic Markov process and apply the previous results to prove the existence of optimal ordinary controls for a tracking problem.

Keywords: Piecewise Deterministic Markov Processes, optimal control, semilinear parabolic equations, dynamic programming, Markov Decision Processes, Optogenetics.
Mathematics Subject Classification: Primary: 93E20, 60J25, 35K58; Secondary: 49L20, 92C20, 92C45.
Citation: Vincent Renault, Michèle Thieullen, Emmanuel Trélat. Optimal control of infinite-dimensional piecewise deterministic Markov processes and application to the control of neuronal dynamics via Optogenetics. Networks & Heterogeneous Media, 2017, 12 (3) : 417-459. doi: 10.3934/nhm.2017019
References:
[1]

N. U. Ahmed, Properties of relaxed trajectories for a class of nonlinear evolution equations on a Banach space, SIAM J. Control Optim., 21 (1983), 953-957.  doi: 10.1137/0321058.  Google Scholar

[2]

N. U. Ahmed and K. L. Teo, Optimal control of systems governed by a class of nonlinear evolution equations in a reflexive Banach space, Journal of Optimization Theory and Applications, 25 (1978), 57-81.   Google Scholar

[3]

N. U. Ahmed and X. Xiang, Properties of relaxed trajectories of evolution equations and optimal control, SIAM J. Control Optim., 31 (1993), 1135-1142.  doi: 10.1137/0331053.  Google Scholar

[4]

T. D. Austin, The emergence of the deterministic Hodgkin-Huxley equations as a limit from the underlying stochastic ion-channel mechanism, Ann. Appl. Probab., 18 (2008), 1279-1325.  doi: 10.1214/07-AAP494.  Google Scholar

[5]

E. J. Balder, A general denseness result for relaxed control theory, Bull. Austral. Math. Soc., 30 (1984), 463-475.  doi: 10.1017/S0004972700002185.  Google Scholar

[6]

D. Bertsekas and S. Shreve, Stochastic Optimal Control: The Discrete-Time Case, Academic Press, 1978.  Google Scholar

[7]

P. Billingsley, Convergence Of Probability Measures, John Wiley & Sons, New York, 1968.  Google Scholar

[8]

E. S. Boyden, F. Zhang, E. Bamberg, G. Nagel and K. Deisseroth, Millisecond-timescale, genetically targeted optical control of neural activity, Nature Neuroscience, 8 (2005), 1263-1268.  doi: 10.1038/nn1525.  Google Scholar

[9]

A. Brandejsky, B. de Saporta and F. Dufour, Numerical methods for the exit time of a Piecewise Deterministic Markov Process, Adv. in Appl. Probab., 44 (2012), 196-225.  doi: 10.1017/S0001867800005504.  Google Scholar

[10]

E. Buckwar and M. Riedler, An exact stochastic hybrid model of excitable membranes including spatio-temporal evolution, J. Math. Biol., 63 (2011), 1051-1093.  doi: 10.1007/s00285-010-0395-z.  Google Scholar

[11]

N. Bäuerle and U. Rieder, Optimal control of Piecewise Deterministic Markov Processes with finite time horizon, Modern Trends of Controlled Stochastic Processes: Theory and Applications, (2010), 144-160.   Google Scholar

[12]

N. Bäuerle and U. Rieder, AMDP algorithms for portfolio optimization problems in pure jump markets, Finance Stoch., 13 (2009), 591-611.  doi: 10.1007/s00780-009-0093-0.  Google Scholar

[13]

N. Bäuerle and U. Rieder, Markov Decision Processes With Applications To Finance, Springer, Heidelberg, 2011. Google Scholar

[14]

O. Costa and F. Dufour, Stability and ergodicity of piecewise deterministic Markov processes, SIAM J. of Control and Opt., 47 (2008), 1053-1077.  doi: 10.1137/060670109.  Google Scholar

[15]

O. Costa and F. Dufour, Singular perturbation for the discounted continuous control of Piecewise Deterministic Markov Processes, Appl. Math. and Opt., 63 (2011), 357-384.  doi: 10.1007/s00245-010-9124-7.  Google Scholar

[16]

O.L.V. Costa, C.A. B Raymundo, F. Dufour and K. Gonzalez, Optimal stopping with continuous control of piecewise deterministic Markov processes, Stoch. Stoch. Rep., 70 (2000), 41-73.  doi: 10.1080/17442500008834245.  Google Scholar

[17]

A. Crudu, A. Debussche, A. Muller and O. Radulescu, Convergence of stochastic gene networks to hybrid piecewise deterministic processe, Ann. Appl. Prob., 22 (2012), 1822-1859.  doi: 10.1214/11-AAP814.  Google Scholar

[18]

M. H. A. Davis, Piecewise-Deterministic Markov Processes: A general class of non-diffusion stochastic models, J. R. Statist. Soc., 46 (1984), 353-388.   Google Scholar

[19]

M. H. A. Davis, Markov Models and Optimization, Chapman and Hall, 1993. doi: 10.1007/978-1-4899-4483-2.  Google Scholar

[20]

B. de Saporta, F. Dufour and H. Zhang, Numerical Methods for Simulation and Optimization of Piecewise Deterministic Markov Processes, Wiley, 2016.  Google Scholar

[21]

J. Diestel and J. J. Uhl, Vector Measures, American Mathematical Society, Providence, 1977.  Google Scholar

[22]

V. Dumas, F. Guillemin and Ph. Robert, A Markovian analysis of additive-increase multiplicative-decrease algorithms, Adv. in Appl. Probab., 34 (2002), 85-111.  doi: 10.1017/S000186780001140X.  Google Scholar

[23]

N. Dunford and J. T. Schwartz, Linear Operators. Part Ⅰ: General Theory, Academic Press, New York-London, 1963.  Google Scholar

[24]

K. -J. Engel and R. Nagel, One Parameter Semigroups for Linear Evolution Equations, Springer-Verlag New York, 2000.  Google Scholar

[25]

M. H. A. Davis, Piecewise deterministic Markov control processes with feedback controls and unbounded costs, Acta Applicandae Mathematicae, 82 (2004), 239-267.  doi: 10.1023/B:ACAP.0000031200.76583.75.  Google Scholar

[26]

R. Gamkrelidze, Principle of Optimal Control Theory Plenum, New York, 1987. Google Scholar

[27]

A. Genadot and M. Thieullen, Averaging for a fully coupled piecewise deterministic Markov process in infinite dimensions, Adv. in Appl. Probab., 44 (2012), 749-773.  doi: 10.1017/S0001867800005863.  Google Scholar

[28]

D. Goreac and M. Martinez, Algebraic invariance conditions in the study of approximate (null-)controllability of Markov switch processes, Mathematics of Control, Signals, and Systems, 27 (2015), 551-578.  doi: 10.1007/s00498-015-0146-1.  Google Scholar

[29]

R. M. Gray and D. L. Neuhoff, Quantization, IEEE Trans. Inform. Theory, 44 (1998), 2325-2383.  doi: 10.1109/18.720541.  Google Scholar

[30]

A. L. Hodgkin and A. F. Huxley, A quantitative description of membrane current and its application to conduction and excitation in nerve, J. Physiol., 117 (1952), 500-544.   Google Scholar

[31]

Q. Hu and W. Yue, Markov Decision Processes with Their Applications, Springer US, 2008.  Google Scholar

[32]

J. Jacod, Multivariate point processes: Predictable projections, Radon-Nikodym derivatives, representation of martingales, Z. Wahrsag. Verw. Gebiete, 34 (1975), 235-253.  doi: 10.1007/BF00536010.  Google Scholar

[33]

K. Nikolic, N. Grossman, M.S. Grubb, J. Burrone, C. Toumazou and P. Degenaar, Photocycles of Channelrhodopsin-2, Photochemistry and Photobiology, 85 (2009), 400-411.  doi: 10.1111/j.1751-1097.2008.00460.x.  Google Scholar

[34]

K. Nikolic, S. Jarvis, N. Grossman and S. Schultz, Computational models of Optogenetic tools for controlling neural circuits with light, Conf. Proc. IEEE Eng. Med. Biol. Soc., (2013), 5934-5937.  doi: 10.1109/EMBC.2013.6610903.  Google Scholar

[35]

G. Pagès, H. Pham and J. Printemps, Handbook of computational and numerical methods in finance, Birkhäuser Boston, (2004), 253-297.   Google Scholar

[36]

K. Pakdaman, M. Thieullen and G. Wainrib, Reduction of stochastic conductance-based neuron models with time-sacles separation, J. Comput. Neurosci., 32 (2012), 327-346.  doi: 10.1007/s10827-011-0355-7.  Google Scholar

[37]

N. S. Papageorgiou, Properties of the relaxed trajectories of evolution equations and optimal control, SIAM J. Control Optim., 27 (1989), 267-288.  doi: 10.1137/0327014.  Google Scholar

[38]

V. Renault, M. Thieullen and E. Trélat, Minimal time spiking in various ChR2-controlled neuron models, J. Math. Biol., (2017), 1-42.  doi: 10.1007/s00285-017-1101-1.  Google Scholar

[39]

M. Riedler, M. Thieullen and G. Wainrib, Limit theorems for infinite-dimensional Piecewise Deterministic Markov Processes. Applications to stochastic excitable membrane models, Electron. J. Probab., 17 (2012), 1-48.   Google Scholar

[40]

D. Vermes, Optimal control of piecewise deterministic Markov processes, Stochastics. An International Journal of Probability and Stochastic Processes, 14 (1985), 165-207.  doi: 10.1080/17442508508833338.  Google Scholar

[41]

J. Warga, Relaxed variational problem, J. Math. Anal. Appl., 4 (1962), 111-128.  doi: 10.1016/0022-247X(62)90033-1.  Google Scholar

[42]

J. Warga, Necessary conditions for minimum in relaxed variational problems, J. Math. Anal. Appl., 4 (1962), 129-145.  doi: 10.1016/0022-247X(62)90034-3.  Google Scholar

[43]

J. Warga, Optimal Control of Differential and Functional Equations, Wiley-Interscience, New York, 1972.  Google Scholar

[44]

J. C. Williams and J. Xu et al, Computational optogenetics: Empirically-derived voltage-and light-sensitive Channelrhodopsin-2 model, JPLoS Comput Biol, 9 (2013), e1003220. doi: 10.1371/journal.pcbi.1003220.  Google Scholar

[45]

L. C. Young, Lectures on the Calculus of Variations and Optimal Control Theory, W. B. Saunders, Philadelphia, PA, 1969.  Google Scholar

[46]

A. A. Yushkevich, On reducing a jump controllable Markov model to a model with discrete time, Theory Probab. Appl., 25 (1980), 58-69.  doi: 10.1137/1125005.  Google Scholar

show all references

References:
[1]

N. U. Ahmed, Properties of relaxed trajectories for a class of nonlinear evolution equations on a Banach space, SIAM J. Control Optim., 21 (1983), 953-957.  doi: 10.1137/0321058.  Google Scholar

[2]

N. U. Ahmed and K. L. Teo, Optimal control of systems governed by a class of nonlinear evolution equations in a reflexive Banach space, Journal of Optimization Theory and Applications, 25 (1978), 57-81.   Google Scholar

[3]

N. U. Ahmed and X. Xiang, Properties of relaxed trajectories of evolution equations and optimal control, SIAM J. Control Optim., 31 (1993), 1135-1142.  doi: 10.1137/0331053.  Google Scholar

[4]

T. D. Austin, The emergence of the deterministic Hodgkin-Huxley equations as a limit from the underlying stochastic ion-channel mechanism, Ann. Appl. Probab., 18 (2008), 1279-1325.  doi: 10.1214/07-AAP494.  Google Scholar

[5]

E. J. Balder, A general denseness result for relaxed control theory, Bull. Austral. Math. Soc., 30 (1984), 463-475.  doi: 10.1017/S0004972700002185.  Google Scholar

[6]

D. Bertsekas and S. Shreve, Stochastic Optimal Control: The Discrete-Time Case, Academic Press, 1978.  Google Scholar

[7]

P. Billingsley, Convergence Of Probability Measures, John Wiley & Sons, New York, 1968.  Google Scholar

[8]

E. S. Boyden, F. Zhang, E. Bamberg, G. Nagel and K. Deisseroth, Millisecond-timescale, genetically targeted optical control of neural activity, Nature Neuroscience, 8 (2005), 1263-1268.  doi: 10.1038/nn1525.  Google Scholar

[9]

A. Brandejsky, B. de Saporta and F. Dufour, Numerical methods for the exit time of a Piecewise Deterministic Markov Process, Adv. in Appl. Probab., 44 (2012), 196-225.  doi: 10.1017/S0001867800005504.  Google Scholar

[10]

E. Buckwar and M. Riedler, An exact stochastic hybrid model of excitable membranes including spatio-temporal evolution, J. Math. Biol., 63 (2011), 1051-1093.  doi: 10.1007/s00285-010-0395-z.  Google Scholar

[11]

N. Bäuerle and U. Rieder, Optimal control of Piecewise Deterministic Markov Processes with finite time horizon, Modern Trends of Controlled Stochastic Processes: Theory and Applications, (2010), 144-160.   Google Scholar

[12]

N. Bäuerle and U. Rieder, AMDP algorithms for portfolio optimization problems in pure jump markets, Finance Stoch., 13 (2009), 591-611.  doi: 10.1007/s00780-009-0093-0.  Google Scholar

[13]

N. Bäuerle and U. Rieder, Markov Decision Processes With Applications To Finance, Springer, Heidelberg, 2011. Google Scholar

[14]

O. Costa and F. Dufour, Stability and ergodicity of piecewise deterministic Markov processes, SIAM J. of Control and Opt., 47 (2008), 1053-1077.  doi: 10.1137/060670109.  Google Scholar

[15]

O. Costa and F. Dufour, Singular perturbation for the discounted continuous control of Piecewise Deterministic Markov Processes, Appl. Math. and Opt., 63 (2011), 357-384.  doi: 10.1007/s00245-010-9124-7.  Google Scholar

[16]

O.L.V. Costa, C.A. B Raymundo, F. Dufour and K. Gonzalez, Optimal stopping with continuous control of piecewise deterministic Markov processes, Stoch. Stoch. Rep., 70 (2000), 41-73.  doi: 10.1080/17442500008834245.  Google Scholar

[17]

A. Crudu, A. Debussche, A. Muller and O. Radulescu, Convergence of stochastic gene networks to hybrid piecewise deterministic processe, Ann. Appl. Prob., 22 (2012), 1822-1859.  doi: 10.1214/11-AAP814.  Google Scholar

[18]

M. H. A. Davis, Piecewise-Deterministic Markov Processes: A general class of non-diffusion stochastic models, J. R. Statist. Soc., 46 (1984), 353-388.   Google Scholar

[19]

M. H. A. Davis, Markov Models and Optimization, Chapman and Hall, 1993. doi: 10.1007/978-1-4899-4483-2.  Google Scholar

[20]

B. de Saporta, F. Dufour and H. Zhang, Numerical Methods for Simulation and Optimization of Piecewise Deterministic Markov Processes, Wiley, 2016.  Google Scholar

[21]

J. Diestel and J. J. Uhl, Vector Measures, American Mathematical Society, Providence, 1977.  Google Scholar

[22]

V. Dumas, F. Guillemin and Ph. Robert, A Markovian analysis of additive-increase multiplicative-decrease algorithms, Adv. in Appl. Probab., 34 (2002), 85-111.  doi: 10.1017/S000186780001140X.  Google Scholar

[23]

N. Dunford and J. T. Schwartz, Linear Operators. Part Ⅰ: General Theory, Academic Press, New York-London, 1963.  Google Scholar

[24]

K. -J. Engel and R. Nagel, One Parameter Semigroups for Linear Evolution Equations, Springer-Verlag New York, 2000.  Google Scholar

[25]

M. H. A. Davis, Piecewise deterministic Markov control processes with feedback controls and unbounded costs, Acta Applicandae Mathematicae, 82 (2004), 239-267.  doi: 10.1023/B:ACAP.0000031200.76583.75.  Google Scholar

[26]

R. Gamkrelidze, Principle of Optimal Control Theory Plenum, New York, 1987. Google Scholar

[27]

A. Genadot and M. Thieullen, Averaging for a fully coupled piecewise deterministic Markov process in infinite dimensions, Adv. in Appl. Probab., 44 (2012), 749-773.  doi: 10.1017/S0001867800005863.  Google Scholar

[28]

D. Goreac and M. Martinez, Algebraic invariance conditions in the study of approximate (null-)controllability of Markov switch processes, Mathematics of Control, Signals, and Systems, 27 (2015), 551-578.  doi: 10.1007/s00498-015-0146-1.  Google Scholar

[29]

R. M. Gray and D. L. Neuhoff, Quantization, IEEE Trans. Inform. Theory, 44 (1998), 2325-2383.  doi: 10.1109/18.720541.  Google Scholar

[30]

A. L. Hodgkin and A. F. Huxley, A quantitative description of membrane current and its application to conduction and excitation in nerve, J. Physiol., 117 (1952), 500-544.   Google Scholar

[31]

Q. Hu and W. Yue, Markov Decision Processes with Their Applications, Springer US, 2008.  Google Scholar

[32]

J. Jacod, Multivariate point processes: Predictable projections, Radon-Nikodym derivatives, representation of martingales, Z. Wahrsag. Verw. Gebiete, 34 (1975), 235-253.  doi: 10.1007/BF00536010.  Google Scholar

[33]

K. Nikolic, N. Grossman, M.S. Grubb, J. Burrone, C. Toumazou and P. Degenaar, Photocycles of Channelrhodopsin-2, Photochemistry and Photobiology, 85 (2009), 400-411.  doi: 10.1111/j.1751-1097.2008.00460.x.  Google Scholar

[34]

K. Nikolic, S. Jarvis, N. Grossman and S. Schultz, Computational models of Optogenetic tools for controlling neural circuits with light, Conf. Proc. IEEE Eng. Med. Biol. Soc., (2013), 5934-5937.  doi: 10.1109/EMBC.2013.6610903.  Google Scholar

[35]

G. Pagès, H. Pham and J. Printemps, Handbook of computational and numerical methods in finance, Birkhäuser Boston, (2004), 253-297.   Google Scholar

[36]

K. Pakdaman, M. Thieullen and G. Wainrib, Reduction of stochastic conductance-based neuron models with time-sacles separation, J. Comput. Neurosci., 32 (2012), 327-346.  doi: 10.1007/s10827-011-0355-7.  Google Scholar

[37]

N. S. Papageorgiou, Properties of the relaxed trajectories of evolution equations and optimal control, SIAM J. Control Optim., 27 (1989), 267-288.  doi: 10.1137/0327014.  Google Scholar

[38]

V. Renault, M. Thieullen and E. Trélat, Minimal time spiking in various ChR2-controlled neuron models, J. Math. Biol., (2017), 1-42.  doi: 10.1007/s00285-017-1101-1.  Google Scholar

[39]

M. Riedler, M. Thieullen and G. Wainrib, Limit theorems for infinite-dimensional Piecewise Deterministic Markov Processes. Applications to stochastic excitable membrane models, Electron. J. Probab., 17 (2012), 1-48.   Google Scholar

[40]

D. Vermes, Optimal control of piecewise deterministic Markov processes, Stochastics. An International Journal of Probability and Stochastic Processes, 14 (1985), 165-207.  doi: 10.1080/17442508508833338.  Google Scholar

[41]

J. Warga, Relaxed variational problem, J. Math. Anal. Appl., 4 (1962), 111-128.  doi: 10.1016/0022-247X(62)90033-1.  Google Scholar

[42]

J. Warga, Necessary conditions for minimum in relaxed variational problems, J. Math. Anal. Appl., 4 (1962), 129-145.  doi: 10.1016/0022-247X(62)90034-3.  Google Scholar

[43]

J. Warga, Optimal Control of Differential and Functional Equations, Wiley-Interscience, New York, 1972.  Google Scholar

[44]

J. C. Williams and J. Xu et al, Computational optogenetics: Empirically-derived voltage-and light-sensitive Channelrhodopsin-2 model, JPLoS Comput Biol, 9 (2013), e1003220. doi: 10.1371/journal.pcbi.1003220.  Google Scholar

[45]

L. C. Young, Lectures on the Calculus of Variations and Optimal Control Theory, W. B. Saunders, Philadelphia, PA, 1969.  Google Scholar

[46]

A. A. Yushkevich, On reducing a jump controllable Markov model to a model with discrete time, Theory Probab. Appl., 25 (1980), 58-69.  doi: 10.1137/1125005.  Google Scholar

Figure 1.  Simplified four states ChR2 channel : $\varepsilon_1$, $\varepsilon_2$, $e_{12}$, $e_{21}$, $K_{d1}$, $K_{d2}$ and $K_r$ are positive constants
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 2.  Simplified ChR2 three states model
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 3.  ChR2 three states model
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 4.  ChR2 channel : $K_{a1}$, $K_{a2}$, and $K_{d2}$ are positive constants defined by:
Figure Options

Download full-size image

Download as PowerPoint slide

Table 1.  Expression of the individual jump rate functions and the Hodgkin-Huxley model
$\underline {{\rm{In}}\;\;{D_1} = \left\{ {{n_0},{n_1},{n_2},{n_3},{n_4}} \right\}} :$
$\sigma_{n_0,n_1}(v,u) = 4\alpha_n(v)$,$\sigma_{n_1,n_2}(v,u) = 3\alpha_n(v)$,
$\sigma_{n_2,n_3}(v,u) = 2\alpha_n(v)$,$\sigma_{n_3,n_4}(v,u) = \alpha_n(v)$
$\sigma_{n_4,n_3}(v,u) = 4\beta_n(v)$,$\sigma_{n_3,n_2}(v,u) = 3\beta_n(v)$,
$\sigma_{n_2,n_1}(v,u) = 2\beta_n(v)$,$\sigma_{n_1,n_0}(v,u) = \beta_n(v)$.
$\underline {{\rm{In}}\;\;{D_2} = \left\{ {{m_0}{h_1},{m_1}{h_1},{m_2}{h_1},{m_3}{h_1},{m_0}{h_0},{m_1}{h_0},{m_2}{h_0},{m_3}{h_0}} \right\}} :$ :
$\sigma_{m_0h_1,m_1h_1}(v,u)=\sigma_{m_0h_0,m_1h_0}(v,u) = 3\alpha_m(v)$,
$\sigma_{m_1h_1,m_2h_1}(v,u) =\sigma_{m_1h_0,m_2h_0}(v,u) = 2\alpha_m(v)$,
$\sigma_{m_2h_1,m_3h_1}(v,u) = \sigma_{m_2h_0,m_3h_0}(v,u) = \alpha_m(v)$,
$\sigma_{m_3h_1,m_2h_1}(v,u) = \sigma_{m_3h_0,m_2h_0}(v,u) = 3\beta_m(v)$,
$\sigma_{m_2h_1,m_1h_1}(v,u) =\sigma_{m_2h_0,m_1h_0}(v,u) = 2\beta_m(v)$,
$\sigma_{m_1h_1,m_0h_1}(v,u) = \sigma_{m_1h_0,m_0h_0}(v,u) = \beta_m(v)$.
$\underline {{\rm{In}}\;\;{D_{ChR2}} = \left\{ {{o_1},{o_2},{c_1},{c_2}} \right\}} :$
$\sigma_{c_1,o_1}(v,u) = \varepsilon_1 u $,$\sigma_{o_1,c_1}(v,u) = K_{d1}$,
$\sigma_{o_1,o_2}(v,u) = e_{12} $,$\sigma_{o_2,o_1}(v,u) = e_{21}$
$\sigma_{o_2,c_2}(v,u) = K_{d2} $,$\sigma_{c_2,o_2}(v,u) = \varepsilon_2 u $,
$\sigma_{c_2,c_1}(v,u) = K_r$.
$\alpha_n(v)=\frac{0.1-0.01v}{e^{1-0.1v}-1}$,$\beta_n(v)=0.125e^{-\frac{v}{80}}$,
$\alpha_m(v)=\frac{2.5-0.1v}{e^{2.5-0.1v}-1}$,$\beta_m(v)=4e^{-\frac{v}{18}}$,
$\alpha_h(v)=0.07e^{-\frac{v}{20}}$,$\beta_h(v)=\frac{1}{e^{3-0.1v}+1}$.
$(HH)\left\{ \begin{aligned} C \dot{V}(t)&= \bar{g}_Kn^4(t)(E_K - V(t)) +\bar{g}_{Na}m^3(t)h(t)(E_{Na}-V(t))\\ & \ \ \ \ \ \ \ \ \ + g_L(E_L-V(t)) + I_{ext}(t),\\ \dot{n}(t)&= \alpha_n(V(t))(1-n(t)) - \beta_n(V(t))n(t),\\ \dot{m}(t)&= \alpha_m(V(t))(1-m(t)) - \beta_m(V(t))m(t),\\ \dot{h}(t)&= \alpha_h(V(t))(1-h(t)) - \beta_h(V(t))h(t). \end{aligned} \right.$
Table Options

Download as excel

$\underline {{\rm{In}}\;\;{D_1} = \left\{ {{n_0},{n_1},{n_2},{n_3},{n_4}} \right\}} :$
$\sigma_{n_0,n_1}(v,u) = 4\alpha_n(v)$,$\sigma_{n_1,n_2}(v,u) = 3\alpha_n(v)$,
$\sigma_{n_2,n_3}(v,u) = 2\alpha_n(v)$,$\sigma_{n_3,n_4}(v,u) = \alpha_n(v)$
$\sigma_{n_4,n_3}(v,u) = 4\beta_n(v)$,$\sigma_{n_3,n_2}(v,u) = 3\beta_n(v)$,
$\sigma_{n_2,n_1}(v,u) = 2\beta_n(v)$,$\sigma_{n_1,n_0}(v,u) = \beta_n(v)$.
$\underline {{\rm{In}}\;\;{D_2} = \left\{ {{m_0}{h_1},{m_1}{h_1},{m_2}{h_1},{m_3}{h_1},{m_0}{h_0},{m_1}{h_0},{m_2}{h_0},{m_3}{h_0}} \right\}} :$ :
$\sigma_{m_0h_1,m_1h_1}(v,u)=\sigma_{m_0h_0,m_1h_0}(v,u) = 3\alpha_m(v)$,
$\sigma_{m_1h_1,m_2h_1}(v,u) =\sigma_{m_1h_0,m_2h_0}(v,u) = 2\alpha_m(v)$,
$\sigma_{m_2h_1,m_3h_1}(v,u) = \sigma_{m_2h_0,m_3h_0}(v,u) = \alpha_m(v)$,
$\sigma_{m_3h_1,m_2h_1}(v,u) = \sigma_{m_3h_0,m_2h_0}(v,u) = 3\beta_m(v)$,
$\sigma_{m_2h_1,m_1h_1}(v,u) =\sigma_{m_2h_0,m_1h_0}(v,u) = 2\beta_m(v)$,
$\sigma_{m_1h_1,m_0h_1}(v,u) = \sigma_{m_1h_0,m_0h_0}(v,u) = \beta_m(v)$.
$\underline {{\rm{In}}\;\;{D_{ChR2}} = \left\{ {{o_1},{o_2},{c_1},{c_2}} \right\}} :$
$\sigma_{c_1,o_1}(v,u) = \varepsilon_1 u $,$\sigma_{o_1,c_1}(v,u) = K_{d1}$,
$\sigma_{o_1,o_2}(v,u) = e_{12} $,$\sigma_{o_2,o_1}(v,u) = e_{21}$
$\sigma_{o_2,c_2}(v,u) = K_{d2} $,$\sigma_{c_2,o_2}(v,u) = \varepsilon_2 u $,
$\sigma_{c_2,c_1}(v,u) = K_r$.
$\alpha_n(v)=\frac{0.1-0.01v}{e^{1-0.1v}-1}$,$\beta_n(v)=0.125e^{-\frac{v}{80}}$,
$\alpha_m(v)=\frac{2.5-0.1v}{e^{2.5-0.1v}-1}$,$\beta_m(v)=4e^{-\frac{v}{18}}$,
$\alpha_h(v)=0.07e^{-\frac{v}{20}}$,$\beta_h(v)=\frac{1}{e^{3-0.1v}+1}$.
$(HH)\left\{ \begin{aligned} C \dot{V}(t)&= \bar{g}_Kn^4(t)(E_K - V(t)) +\bar{g}_{Na}m^3(t)h(t)(E_{Na}-V(t))\\ & \ \ \ \ \ \ \ \ \ + g_L(E_L-V(t)) + I_{ext}(t),\\ \dot{n}(t)&= \alpha_n(V(t))(1-n(t)) - \beta_n(V(t))n(t),\\ \dot{m}(t)&= \alpha_m(V(t))(1-m(t)) - \beta_m(V(t))m(t),\\ \dot{h}(t)&= \alpha_h(V(t))(1-h(t)) - \beta_h(V(t))h(t). \end{aligned} \right.$
[1]

Mathias Staudigl. A limit theorem for Markov decision processes. Journal of Dynamics & Games, 2014, 1 (4) : 639-659. doi: 10.3934/jdg.2014.1.639
[2]

Artur Stephan, Holger Stephan. Memory equations as reduced Markov processes. Discrete & Continuous Dynamical Systems, 2019, 39 (4) : 2133-2155. doi: 10.3934/dcds.2019089
[3]

A. Mittal, N. Hemachandra. Learning algorithms for finite horizon constrained Markov decision processes. Journal of Industrial & Management Optimization, 2007, 3 (3) : 429-444. doi: 10.3934/jimo.2007.3.429
[4]

Wael Bahsoun, Paweł Góra. SRB measures for certain Markov processes. Discrete & Continuous Dynamical Systems, 2011, 30 (1) : 17-37. doi: 10.3934/dcds.2011.30.17
[5]

Qiuli Liu, Xiaolong Zou. A risk minimization problem for finite horizon semi-Markov decision processes with loss rates. Journal of Dynamics & Games, 2018, 5 (2) : 143-163. doi: 10.3934/jdg.2018009
[6]

H.Thomas Banks, Shuhua Hu. Nonlinear stochastic Markov processes and modeling uncertainty in populations. Mathematical Biosciences & Engineering, 2012, 9 (1) : 1-25. doi: 10.3934/mbe.2012.9.1
[7]

Angelica Pachon, Federico Polito, Costantino Ricciuti. On discrete-time semi-Markov processes. Discrete & Continuous Dynamical Systems - B, 2021, 26 (3) : 1499-1529. doi: 10.3934/dcdsb.2020170
[8]

Thomas Kruse, Mikhail Urusov. Approximating exit times of continuous Markov processes. Discrete & Continuous Dynamical Systems - B, 2020, 25 (9) : 3631-3650. doi: 10.3934/dcdsb.2020076
[9]

Xian Chen, Zhi-Ming Ma. A transformation of Markov jump processes and applications in genetic study. Discrete & Continuous Dynamical Systems, 2014, 34 (12) : 5061-5084. doi: 10.3934/dcds.2014.34.5061
[10]

A. M. Vershik. Polymorphisms, Markov processes, quasi-similarity. Discrete & Continuous Dynamical Systems, 2005, 13 (5) : 1305-1324. doi: 10.3934/dcds.2005.13.1305
[11]

Vladimir Kazakov. Sampling - reconstruction procedure with jitter of markov continuous processes formed by stochastic differential equations of the first order. Conference Publications, 2009, 2009 (Special) : 433-441. doi: 10.3934/proc.2009.2009.433
[12]

Oliver Knill. A deterministic displacement theorem for Poisson processes. Electronic Research Announcements, 1997, 3: 110-113.

[13]

Marzia Bisi, Maria Groppi, Giorgio Martalò, Romina Travaglini. Optimal control of leachate recirculation for anaerobic processes in landfills. Discrete & Continuous Dynamical Systems - B, 2021, 26 (6) : 2957-2976. doi: 10.3934/dcdsb.2020215
[14]

Roberto C. Cabrales, Gema Camacho, Enrique Fernández-Cara. Analysis and optimal control of some solidification processes. Discrete & Continuous Dynamical Systems, 2014, 34 (10) : 3985-4017. doi: 10.3934/dcds.2014.34.3985
[15]

Kamil Aida-Zade, Jamila Asadova. Numerical solution to optimal control problems of oscillatory processes. Journal of Industrial & Management Optimization, 2021  doi: 10.3934/jimo.2021166
[16]

Zhenzhong Zhang, Enhua Zhang, Jinying Tong. Necessary and sufficient conditions for ergodicity of CIR model driven by stable processes with Markov switching. Discrete & Continuous Dynamical Systems - B, 2018, 23 (6) : 2433-2455. doi: 10.3934/dcdsb.2018053
[17]

Armen Shirikyan. Ergodicity for a class of Markov processes and applications to randomly forced PDE'S. II. Discrete & Continuous Dynamical Systems - B, 2006, 6 (4) : 911-926. doi: 10.3934/dcdsb.2006.6.911
[18]

Gábor Horváth, Zsolt Saffer, Miklós Telek. Queue length analysis of a Markov-modulated vacation queue with dependent arrival and service processes and exhaustive service policy. Journal of Industrial & Management Optimization, 2017, 13 (3) : 1365-1381. doi: 10.3934/jimo.2016077
[19]

Linyi Qian, Wei Wang, Rongming Wang. Risk-minimizing portfolio selection for insurance payment processes under a Markov-modulated model. Journal of Industrial & Management Optimization, 2013, 9 (2) : 411-429. doi: 10.3934/jimo.2013.9.411
[20]

Rein Luus. Optimal control of oscillatory systems by iterative dynamic programming. Journal of Industrial & Management Optimization, 2008, 4 (1) : 1-15. doi: 10.3934/jimo.2008.4.1

2020 Impact Factor: 1.213

Tools

Metrics

  • PDF downloads (62)
  • HTML views (61)
  • Cited by (2)

Other articles
by authors

[Back to Top]

Copyright © 2021 American Institute of Mathematical Sciences