# American Institute of Mathematical Sciences

September  2017, 12(3): 417-459. doi: 10.3934/nhm.2017019

## Optimal control of infinite-dimensional piecewise deterministic Markov processes and application to the control of neuronal dynamics via Optogenetics

 1 Sorbonne Universités, UPMC Univ Paris 06, CNRS UMR 7599, Laboratoire de Probabilités et Modèles Aléatoires, F-75005, Paris, France 2 Sorbonne Universités, UPMC Univ Paris 06, CNRS UMR 7598, Laboratoire Jacques-Louis Lions, F-75005, Paris, France

Received  July 2016 Revised  May 2017 Published  September 2017

In this paper we define an infinite-dimensional controlled piecewise deterministic Markov process (PDMP) and we study an optimal control problem with finite time horizon and unbounded cost. This process is a coupling between a continuous time Markov Chain and a set of semilinear parabolic partial differential equations, both processes depending on the control. We apply dynamic programming to the embedded Markov decision process to obtain existence of optimal relaxed controls and we give some sufficient conditions ensuring the existence of an optimal ordinary control. This study, which constitutes an extension of controlled PDMPs to infinite dimension, is motivated by the control that provides Optogenetics on neuron models such as the Hodgkin-Huxley model. We define an infinite-dimensional controlled Hodgkin-Huxley model as an infinite-dimensional controlled piecewise deterministic Markov process and apply the previous results to prove the existence of optimal ordinary controls for a tracking problem.

Citation: Vincent Renault, Michèle Thieullen, Emmanuel Trélat. Optimal control of infinite-dimensional piecewise deterministic Markov processes and application to the control of neuronal dynamics via Optogenetics. Networks & Heterogeneous Media, 2017, 12 (3) : 417-459. doi: 10.3934/nhm.2017019
##### References:

show all references

##### References:
Simplified four states ChR2 channel : $\varepsilon_1$, $\varepsilon_2$, $e_{12}$, $e_{21}$, $K_{d1}$, $K_{d2}$ and $K_r$ are positive constants
Simplified ChR2 three states model
ChR2 three states model
ChR2 channel : $K_{a1}$, $K_{a2}$, and $K_{d2}$ are positive constants defined by:
Expression of the individual jump rate functions and the Hodgkin-Huxley model
 $\underline {{\rm{In}}\;\;{D_1} = \left\{ {{n_0},{n_1},{n_2},{n_3},{n_4}} \right\}} :$ $\sigma_{n_0,n_1}(v,u) = 4\alpha_n(v)$, $\sigma_{n_1,n_2}(v,u) = 3\alpha_n(v)$, $\sigma_{n_2,n_3}(v,u) = 2\alpha_n(v)$, $\sigma_{n_3,n_4}(v,u) = \alpha_n(v)$ $\sigma_{n_4,n_3}(v,u) = 4\beta_n(v)$, $\sigma_{n_3,n_2}(v,u) = 3\beta_n(v)$, $\sigma_{n_2,n_1}(v,u) = 2\beta_n(v)$, $\sigma_{n_1,n_0}(v,u) = \beta_n(v)$. $\underline {{\rm{In}}\;\;{D_2} = \left\{ {{m_0}{h_1},{m_1}{h_1},{m_2}{h_1},{m_3}{h_1},{m_0}{h_0},{m_1}{h_0},{m_2}{h_0},{m_3}{h_0}} \right\}} :$ : $\sigma_{m_0h_1,m_1h_1}(v,u)=\sigma_{m_0h_0,m_1h_0}(v,u) = 3\alpha_m(v)$, $\sigma_{m_1h_1,m_2h_1}(v,u) =\sigma_{m_1h_0,m_2h_0}(v,u) = 2\alpha_m(v)$, $\sigma_{m_2h_1,m_3h_1}(v,u) = \sigma_{m_2h_0,m_3h_0}(v,u) = \alpha_m(v)$, $\sigma_{m_3h_1,m_2h_1}(v,u) = \sigma_{m_3h_0,m_2h_0}(v,u) = 3\beta_m(v)$, $\sigma_{m_2h_1,m_1h_1}(v,u) =\sigma_{m_2h_0,m_1h_0}(v,u) = 2\beta_m(v)$, $\sigma_{m_1h_1,m_0h_1}(v,u) = \sigma_{m_1h_0,m_0h_0}(v,u) = \beta_m(v)$. $\underline {{\rm{In}}\;\;{D_{ChR2}} = \left\{ {{o_1},{o_2},{c_1},{c_2}} \right\}} :$ $\sigma_{c_1,o_1}(v,u) = \varepsilon_1 u$, $\sigma_{o_1,c_1}(v,u) = K_{d1}$, $\sigma_{o_1,o_2}(v,u) = e_{12}$, $\sigma_{o_2,o_1}(v,u) = e_{21}$ $\sigma_{o_2,c_2}(v,u) = K_{d2}$, $\sigma_{c_2,o_2}(v,u) = \varepsilon_2 u$, $\sigma_{c_2,c_1}(v,u) = K_r$. $\alpha_n(v)=\frac{0.1-0.01v}{e^{1-0.1v}-1}$, $\beta_n(v)=0.125e^{-\frac{v}{80}}$, $\alpha_m(v)=\frac{2.5-0.1v}{e^{2.5-0.1v}-1}$, $\beta_m(v)=4e^{-\frac{v}{18}}$, $\alpha_h(v)=0.07e^{-\frac{v}{20}}$, $\beta_h(v)=\frac{1}{e^{3-0.1v}+1}$. (HH)\left\{ \begin{aligned} C \dot{V}(t)&= \bar{g}_Kn^4(t)(E_K - V(t)) +\bar{g}_{Na}m^3(t)h(t)(E_{Na}-V(t))\\ & \ \ \ \ \ \ \ \ \ + g_L(E_L-V(t)) + I_{ext}(t),\\ \dot{n}(t)&= \alpha_n(V(t))(1-n(t)) - \beta_n(V(t))n(t),\\ \dot{m}(t)&= \alpha_m(V(t))(1-m(t)) - \beta_m(V(t))m(t),\\ \dot{h}(t)&= \alpha_h(V(t))(1-h(t)) - \beta_h(V(t))h(t). \end{aligned} \right.
 $\underline {{\rm{In}}\;\;{D_1} = \left\{ {{n_0},{n_1},{n_2},{n_3},{n_4}} \right\}} :$ $\sigma_{n_0,n_1}(v,u) = 4\alpha_n(v)$, $\sigma_{n_1,n_2}(v,u) = 3\alpha_n(v)$, $\sigma_{n_2,n_3}(v,u) = 2\alpha_n(v)$, $\sigma_{n_3,n_4}(v,u) = \alpha_n(v)$ $\sigma_{n_4,n_3}(v,u) = 4\beta_n(v)$, $\sigma_{n_3,n_2}(v,u) = 3\beta_n(v)$, $\sigma_{n_2,n_1}(v,u) = 2\beta_n(v)$, $\sigma_{n_1,n_0}(v,u) = \beta_n(v)$. $\underline {{\rm{In}}\;\;{D_2} = \left\{ {{m_0}{h_1},{m_1}{h_1},{m_2}{h_1},{m_3}{h_1},{m_0}{h_0},{m_1}{h_0},{m_2}{h_0},{m_3}{h_0}} \right\}} :$ : $\sigma_{m_0h_1,m_1h_1}(v,u)=\sigma_{m_0h_0,m_1h_0}(v,u) = 3\alpha_m(v)$, $\sigma_{m_1h_1,m_2h_1}(v,u) =\sigma_{m_1h_0,m_2h_0}(v,u) = 2\alpha_m(v)$, $\sigma_{m_2h_1,m_3h_1}(v,u) = \sigma_{m_2h_0,m_3h_0}(v,u) = \alpha_m(v)$, $\sigma_{m_3h_1,m_2h_1}(v,u) = \sigma_{m_3h_0,m_2h_0}(v,u) = 3\beta_m(v)$, $\sigma_{m_2h_1,m_1h_1}(v,u) =\sigma_{m_2h_0,m_1h_0}(v,u) = 2\beta_m(v)$, $\sigma_{m_1h_1,m_0h_1}(v,u) = \sigma_{m_1h_0,m_0h_0}(v,u) = \beta_m(v)$. $\underline {{\rm{In}}\;\;{D_{ChR2}} = \left\{ {{o_1},{o_2},{c_1},{c_2}} \right\}} :$ $\sigma_{c_1,o_1}(v,u) = \varepsilon_1 u$, $\sigma_{o_1,c_1}(v,u) = K_{d1}$, $\sigma_{o_1,o_2}(v,u) = e_{12}$, $\sigma_{o_2,o_1}(v,u) = e_{21}$ $\sigma_{o_2,c_2}(v,u) = K_{d2}$, $\sigma_{c_2,o_2}(v,u) = \varepsilon_2 u$, $\sigma_{c_2,c_1}(v,u) = K_r$. $\alpha_n(v)=\frac{0.1-0.01v}{e^{1-0.1v}-1}$, $\beta_n(v)=0.125e^{-\frac{v}{80}}$, $\alpha_m(v)=\frac{2.5-0.1v}{e^{2.5-0.1v}-1}$, $\beta_m(v)=4e^{-\frac{v}{18}}$, $\alpha_h(v)=0.07e^{-\frac{v}{20}}$, $\beta_h(v)=\frac{1}{e^{3-0.1v}+1}$. (HH)\left\{ \begin{aligned} C \dot{V}(t)&= \bar{g}_Kn^4(t)(E_K - V(t)) +\bar{g}_{Na}m^3(t)h(t)(E_{Na}-V(t))\\ & \ \ \ \ \ \ \ \ \ + g_L(E_L-V(t)) + I_{ext}(t),\\ \dot{n}(t)&= \alpha_n(V(t))(1-n(t)) - \beta_n(V(t))n(t),\\ \dot{m}(t)&= \alpha_m(V(t))(1-m(t)) - \beta_m(V(t))m(t),\\ \dot{h}(t)&= \alpha_h(V(t))(1-h(t)) - \beta_h(V(t))h(t). \end{aligned} \right.
 [1] Marzia Bisi, Maria Groppi, Giorgio Martalò, Romina Travaglini. Optimal control of leachate recirculation for anaerobic processes in landfills. Discrete & Continuous Dynamical Systems - B, 2021, 26 (6) : 2957-2976. doi: 10.3934/dcdsb.2020215 [2] Kehan Si, Zhenda Xu, Ka Fai Cedric Yiu, Xun Li. Open-loop solvability for mean-field stochastic linear quadratic optimal control problems of Markov regime-switching system. Journal of Industrial & Management Optimization, 2021  doi: 10.3934/jimo.2021074 [3] Flank D. M. Bezerra, Jacson Simsen, Mariza Stefanello Simsen. Convergence of quasilinear parabolic equations to semilinear equations. Discrete & Continuous Dynamical Systems - B, 2021, 26 (7) : 3823-3834. doi: 10.3934/dcdsb.2020258 [4] Luke Finlay, Vladimir Gaitsgory, Ivan Lebedev. Linear programming solutions of periodic optimization problems: approximation of the optimal control. Journal of Industrial & Management Optimization, 2007, 3 (2) : 399-413. doi: 10.3934/jimo.2007.3.399 [5] Ajay Jasra, Kody J. H. Law, Yaxian Xu. Markov chain simulation for multilevel Monte Carlo. Foundations of Data Science, 2021, 3 (1) : 27-47. doi: 10.3934/fods.2021004 [6] Wenyuan Wang, Ran Xu. General drawdown based dividend control with fixed transaction costs for spectrally negative Lévy risk processes. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2020179 [7] Vladimir Gaitsgory, Ilya Shvartsman. Linear programming estimates for Cesàro and Abel limits of optimal values in optimal control problems. Discrete & Continuous Dynamical Systems - B, 2021  doi: 10.3934/dcdsb.2021102 [8] Lakmi Niwanthi Wadippuli, Ivan Gudoshnikov, Oleg Makarenkov. Global asymptotic stability of nonconvex sweeping processes. Discrete & Continuous Dynamical Systems - B, 2020, 25 (3) : 1129-1139. doi: 10.3934/dcdsb.2019212 [9] Liqiang Jin, Yanqing Liu, Yanyan Yin, Kok Lay Teo, Fei Liu. Design of probabilistic $l_2-l_\infty$ filter for uncertain Markov jump systems with partial information of the transition probabilities. Journal of Industrial & Management Optimization, 2021  doi: 10.3934/jimo.2021070 [10] Matheus C. Bortolan, José Manuel Uzal. Upper and weak-lower semicontinuity of pullback attractors to impulsive evolution processes. Discrete & Continuous Dynamical Systems - B, 2021, 26 (7) : 3667-3692. doi: 10.3934/dcdsb.2020252 [11] Mingshang Hu, Shige Peng. G-Lévy processes under sublinear expectations. Probability, Uncertainty and Quantitative Risk, 2021, 6 (1) : 1-22. doi: 10.3934/puqr.2021001 [12] Mohammed Abdelghany, Amr B. Eltawil, Zakaria Yahia, Kazuhide Nakata. A hybrid variable neighbourhood search and dynamic programming approach for the nurse rostering problem. Journal of Industrial & Management Optimization, 2021, 17 (4) : 2051-2072. doi: 10.3934/jimo.2020058 [13] Meiqiao Ai, Zhimin Zhang, Wenguang Yu. First passage problems of refracted jump diffusion processes and their applications in valuing equity-linked death benefits. Journal of Industrial & Management Optimization, 2021  doi: 10.3934/jimo.2021039 [14] Shanjian Tang, Fu Zhang. Path-dependent optimal stochastic control and viscosity solution of associated Bellman equations. Discrete & Continuous Dynamical Systems, 2015, 35 (11) : 5521-5553. doi: 10.3934/dcds.2015.35.5521 [15] Chih-Chiang Fang. Bayesian decision making in determining optimal leased term and preventive maintenance scheme for leased facilities. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2020127 [16] Nikolaos Roidos. Expanding solutions of quasilinear parabolic equations. Communications on Pure & Applied Analysis, , () : -. doi: 10.3934/cpaa.2021026 [17] María J. Garrido-Atienza, Bohdan Maslowski, Jana  Šnupárková. Semilinear stochastic equations with bilinear fractional noise. Discrete & Continuous Dynamical Systems - B, 2016, 21 (9) : 3075-3094. doi: 10.3934/dcdsb.2016088 [18] Tobias Geiger, Daniel Wachsmuth, Gerd Wachsmuth. Optimal control of ODEs with state suprema. Mathematical Control & Related Fields, 2021  doi: 10.3934/mcrf.2021012 [19] Mohamed Ouzahra. Approximate controllability of the semilinear reaction-diffusion equation governed by a multiplicative control. Discrete & Continuous Dynamical Systems - B, 2021  doi: 10.3934/dcdsb.2021081 [20] Qiang Guo, Dong Liang. An adaptive wavelet method and its analysis for parabolic equations. Numerical Algebra, Control & Optimization, 2013, 3 (2) : 327-345. doi: 10.3934/naco.2013.3.327

2019 Impact Factor: 1.053