• Previous Article
    Continuous dependence and optimal control of a dynamic elastic-viscoplastic contact problem with non-monotone boundary conditions
  • EECT Home
  • This Issue
  • Next Article
    Invariant measure for neutral stochastic functional differential equations with non-Lipschitz coefficients
doi: 10.3934/eect.2022023
Online First

Online First articles are published articles within a journal that have not yet been assigned to a formal issue. This means they do not yet have a volume number, issue number, or page numbers assigned to them, however, they can still be found and cited using their DOI (Digital Object Identifier). Online First publication benefits the research community by making new scientific discoveries known as quickly as possible.

Readers can access Online First articles via the “Online First” tab for the selected journal.

Optimal control for stochastic differential equations and related Kolmogorov equations

1. 

"Octav Mayer" Institute of Mathematics of the Romanian Academy, Iași 700506

2. 

"Simion Stoilow" Institute of Mathematics of the Romanian Academy, Bucharest 014700, Romania

Received  August 2021 Revised  March 2022 Early access April 2022

This paper concerns a stochastic optimal control problem with feedback Markov inputs. The problem is reduced to a deterministic optimal control problem for a Kolmogorov equation where the control for the deterministic problem is of open-loop type. The existence of an optimal control is proved for the deterministic control problem in a particular case. A maximum principle and some first order necessary optimality conditions are derived. Some examples and comments are discussed.

Citation: Ștefana-Lucia Aniţa. Optimal control for stochastic differential equations and related Kolmogorov equations. Evolution Equations and Control Theory, doi: 10.3934/eect.2022023
References:
[1]

Ș.-L. Aniţa, A stochastic optimal control problem with feedback inputs, Int. J. Control, 95 (2022), 589-602.  doi: 10.1080/00207179.2020.1806360.

[2]

Ș.-L. Aniţa, Optimal control of stochastic differential equations via Fokker-Planck equations, Appl. Math. Optim., 84 (2021), 1555-1583.  doi: 10.1007/s00245-021-09804-5.

[3]

M. Annunziato and A. Borzi, A Fokker-Planck control framework for multidimensional stochastic processes, J. Comput. Appl. Math., 237 (2013), 487-507.  doi: 10.1016/j.cam.2012.06.019.

[4]

M. Annunziato and A. Borzi, A Fokker-Planck control framework for stochastic systems, EMS Surv. Math. Sci., 5 (2018), 65-98.  doi: 10.4171/EMSS/27.

[5]

V. Barbu, Mathematical Methods in Optimization of Differential Systems, Springer, Dordrecht, 1994. doi: 10.1007/978-94-011-0760-0.

[6]

V. Barbu, Nonlinear Differential Equations of Monotone Types in Banach Spaces, Springer-Verlag, New York, 2010. doi: 10.1007/978-1-4419-5542-5.

[7]

V. Barbu, Optimal feedback controllers for a stochastic differential equation with reflection, SIAM J. Control Optim., 58 (2020), 986-997.  doi: 10.1137/19M1294423.

[8]

V. BarbuC. Benazzoli and L. Di Persio, Feedback optimal controllers for the Heston model, Appl. Math. Optim., 81 (2020), 739-756.  doi: 10.1007/s00245-018-9517-6.

[9]

V. Barbu, M. Röckner and D. Zhang, Stochastic differential equations with variable structure driven by multiplicative Gaussian noise and sliding mode dynamics, SIAM J. Control Optim., 58 (2020), 2383–2410.

[10]

H. Brézis, Analyse Fonctionnelle. Théorie et Applications, Dunod, Paris, 2005.

[11]

G. Da Prato, Kolmogorov Equations for Stochastic PDEs, Birkhäuser Verlag, Basel, 2004. doi: 10.1007/978-3-0348-7909-5.

[12]

L. C. Evans, An Introduction to Stochastic Differential Equations, American Mathematical Society, 2013. doi: 10.1090/mbk/082.

[13]

A. Fleig and R. Gugliemi, Optimal control of the Fokker-Planck equation with space-dependent controls, J. Optim. Theory Appl., 174 (2017), 408-427.  doi: 10.1007/s10957-017-1120-5.

[14]

W. H. Fleming and R. W. Rishel, Deterministic and Stochastic Optimal Control, Springer-Verlag, New York, 1975.

[15]

A. Friedman, Stochastic Differential Equations and Applications, Dover, New York, 2006.

[16]

N. V. Krylov, On Kolmogorov's equations for finite-dimensional diffusions, In Stochastic PDEs and Kolmogorov Equations in Infinite Dimensions, (Cetraro 1998), (ed. G. Da Prato), Springer, Berlin, 1715 (1999), 1–63. doi: 10.1007/BFb0092417.

[17]

X. Li and J. Yong, Optimal Control Theory for Infinite Dimensional Systems, Birkhäuser, Boston, 1995. doi: 10.1007/978-1-4612-4260-4.

[18]

J. -L. Lions, Quelques Méthodes de Résolution des Problèmes Aux Limites Nonlinéaires, Dunod, Gauthier-Villars, Paris, 1969.

[19]

B. Øksendal, Stochastic Differential Equations. An Introduction with Applications, 6th edition, Springer, Berlin, 2003. doi: 10.1007/978-3-642-14394-6.

[20]

A. Pazy, Semigroups of Linear Operators and Applications to Partial Differential Equations, Springer-Verlag, New York, 1983. doi: 10.1007/978-1-4612-5561-1.

[21]

M. I. Sumin, The first variation and Pontryagin's maximum principle in optimal control for partial differential equations, Comput. Math. Math. Phys., 49 (2009), 958-978.  doi: 10.1134/S0965542509060062.

show all references

References:
[1]

Ș.-L. Aniţa, A stochastic optimal control problem with feedback inputs, Int. J. Control, 95 (2022), 589-602.  doi: 10.1080/00207179.2020.1806360.

[2]

Ș.-L. Aniţa, Optimal control of stochastic differential equations via Fokker-Planck equations, Appl. Math. Optim., 84 (2021), 1555-1583.  doi: 10.1007/s00245-021-09804-5.

[3]

M. Annunziato and A. Borzi, A Fokker-Planck control framework for multidimensional stochastic processes, J. Comput. Appl. Math., 237 (2013), 487-507.  doi: 10.1016/j.cam.2012.06.019.

[4]

M. Annunziato and A. Borzi, A Fokker-Planck control framework for stochastic systems, EMS Surv. Math. Sci., 5 (2018), 65-98.  doi: 10.4171/EMSS/27.

[5]

V. Barbu, Mathematical Methods in Optimization of Differential Systems, Springer, Dordrecht, 1994. doi: 10.1007/978-94-011-0760-0.

[6]

V. Barbu, Nonlinear Differential Equations of Monotone Types in Banach Spaces, Springer-Verlag, New York, 2010. doi: 10.1007/978-1-4419-5542-5.

[7]

V. Barbu, Optimal feedback controllers for a stochastic differential equation with reflection, SIAM J. Control Optim., 58 (2020), 986-997.  doi: 10.1137/19M1294423.

[8]

V. BarbuC. Benazzoli and L. Di Persio, Feedback optimal controllers for the Heston model, Appl. Math. Optim., 81 (2020), 739-756.  doi: 10.1007/s00245-018-9517-6.

[9]

V. Barbu, M. Röckner and D. Zhang, Stochastic differential equations with variable structure driven by multiplicative Gaussian noise and sliding mode dynamics, SIAM J. Control Optim., 58 (2020), 2383–2410.

[10]

H. Brézis, Analyse Fonctionnelle. Théorie et Applications, Dunod, Paris, 2005.

[11]

G. Da Prato, Kolmogorov Equations for Stochastic PDEs, Birkhäuser Verlag, Basel, 2004. doi: 10.1007/978-3-0348-7909-5.

[12]

L. C. Evans, An Introduction to Stochastic Differential Equations, American Mathematical Society, 2013. doi: 10.1090/mbk/082.

[13]

A. Fleig and R. Gugliemi, Optimal control of the Fokker-Planck equation with space-dependent controls, J. Optim. Theory Appl., 174 (2017), 408-427.  doi: 10.1007/s10957-017-1120-5.

[14]

W. H. Fleming and R. W. Rishel, Deterministic and Stochastic Optimal Control, Springer-Verlag, New York, 1975.

[15]

A. Friedman, Stochastic Differential Equations and Applications, Dover, New York, 2006.

[16]

N. V. Krylov, On Kolmogorov's equations for finite-dimensional diffusions, In Stochastic PDEs and Kolmogorov Equations in Infinite Dimensions, (Cetraro 1998), (ed. G. Da Prato), Springer, Berlin, 1715 (1999), 1–63. doi: 10.1007/BFb0092417.

[17]

X. Li and J. Yong, Optimal Control Theory for Infinite Dimensional Systems, Birkhäuser, Boston, 1995. doi: 10.1007/978-1-4612-4260-4.

[18]

J. -L. Lions, Quelques Méthodes de Résolution des Problèmes Aux Limites Nonlinéaires, Dunod, Gauthier-Villars, Paris, 1969.

[19]

B. Øksendal, Stochastic Differential Equations. An Introduction with Applications, 6th edition, Springer, Berlin, 2003. doi: 10.1007/978-3-642-14394-6.

[20]

A. Pazy, Semigroups of Linear Operators and Applications to Partial Differential Equations, Springer-Verlag, New York, 1983. doi: 10.1007/978-1-4612-5561-1.

[21]

M. I. Sumin, The first variation and Pontryagin's maximum principle in optimal control for partial differential equations, Comput. Math. Math. Phys., 49 (2009), 958-978.  doi: 10.1134/S0965542509060062.

[1]

Gökçe Dİlek Küçük, Gabil Yagub, Ercan Çelİk. On the existence and uniqueness of the solution of an optimal control problem for Schrödinger equation. Discrete and Continuous Dynamical Systems - S, 2019, 12 (3) : 503-512. doi: 10.3934/dcdss.2019033

[2]

Zhen-Zhen Tao, Bing Sun. A feedback design for numerical solution to optimal control problems based on Hamilton-Jacobi-Bellman equation. Electronic Research Archive, 2021, 29 (5) : 3429-3447. doi: 10.3934/era.2021046

[3]

Nguyen Thi Hoai. Asymptotic approximation to a solution of a singularly perturbed linear-quadratic optimal control problem with second-order linear ordinary differential equation of state variable. Numerical Algebra, Control and Optimization, 2021, 11 (4) : 495-512. doi: 10.3934/naco.2020040

[4]

V.N. Malozemov, A.V. Omelchenko. On a discrete optimal control problem with an explicit solution. Journal of Industrial and Management Optimization, 2006, 2 (1) : 55-62. doi: 10.3934/jimo.2006.2.55

[5]

Sie Long Kek, Kok Lay Teo, Mohd Ismail Abd Aziz. Filtering solution of nonlinear stochastic optimal control problem in discrete-time with model-reality differences. Numerical Algebra, Control and Optimization, 2012, 2 (1) : 207-222. doi: 10.3934/naco.2012.2.207

[6]

Diana Keller. Optimal control of a linear stochastic Schrödinger equation. Conference Publications, 2013, 2013 (special) : 437-446. doi: 10.3934/proc.2013.2013.437

[7]

Fulvia Confortola, Elisa Mastrogiacomo. Optimal control for stochastic heat equation with memory. Evolution Equations and Control Theory, 2014, 3 (1) : 35-58. doi: 10.3934/eect.2014.3.35

[8]

Marcus Wagner. A direct method for the solution of an optimal control problem arising from image registration. Numerical Algebra, Control and Optimization, 2012, 2 (3) : 487-510. doi: 10.3934/naco.2012.2.487

[9]

Canghua Jiang, Kok Lay Teo, Ryan Loxton, Guang-Ren Duan. A neighboring extremal solution for an optimal switched impulsive control problem. Journal of Industrial and Management Optimization, 2012, 8 (3) : 591-609. doi: 10.3934/jimo.2012.8.591

[10]

Alexander Arguchintsev, Vasilisa Poplevko. An optimal control problem by parabolic equation with boundary smooth control and an integral constraint. Numerical Algebra, Control and Optimization, 2018, 8 (2) : 193-202. doi: 10.3934/naco.2018011

[11]

Kamil Aida-Zade, Jamila Asadova. Numerical solution to optimal control problems of oscillatory processes. Journal of Industrial and Management Optimization, 2021  doi: 10.3934/jimo.2021166

[12]

Shihchung Chiang. Numerical optimal unbounded control with a singular integro-differential equation as a constraint. Conference Publications, 2013, 2013 (special) : 129-137. doi: 10.3934/proc.2013.2013.129

[13]

Lukáš Adam, Jiří Outrata. On optimal control of a sweeping process coupled with an ordinary differential equation. Discrete and Continuous Dynamical Systems - B, 2014, 19 (9) : 2709-2738. doi: 10.3934/dcdsb.2014.19.2709

[14]

Jan-Hendrik Webert, Philip E. Gill, Sven-Joachim Kimmerle, Matthias Gerdts. A study of structure-exploiting SQP algorithms for an optimal control problem with coupled hyperbolic and ordinary differential equation constraints. Discrete and Continuous Dynamical Systems - S, 2018, 11 (6) : 1259-1282. doi: 10.3934/dcdss.2018071

[15]

Haiyang Wang, Zhen Wu. Time-inconsistent optimal control problem with random coefficients and stochastic equilibrium HJB equation. Mathematical Control and Related Fields, 2015, 5 (3) : 651-678. doi: 10.3934/mcrf.2015.5.651

[16]

Shanjian Tang, Fu Zhang. Path-dependent optimal stochastic control and viscosity solution of associated Bellman equations. Discrete and Continuous Dynamical Systems, 2015, 35 (11) : 5521-5553. doi: 10.3934/dcds.2015.35.5521

[17]

Qing Tang. On an optimal control problem of time-fractional advection-diffusion equation. Discrete and Continuous Dynamical Systems - B, 2020, 25 (2) : 761-779. doi: 10.3934/dcdsb.2019266

[18]

Peter I. Kogut. On approximation of an optimal boundary control problem for linear elliptic equation with unbounded coefficients. Discrete and Continuous Dynamical Systems, 2014, 34 (5) : 2105-2133. doi: 10.3934/dcds.2014.34.2105

[19]

Max Gunzburger, Sung-Dae Yang, Wenxiang Zhu. Analysis and discretization of an optimal control problem for the forced Fisher equation. Discrete and Continuous Dynamical Systems - B, 2007, 8 (3) : 569-587. doi: 10.3934/dcdsb.2007.8.569

[20]

Fulvia Confortola, Elisa Mastrogiacomo. Feedback optimal control for stochastic Volterra equations with completely monotone kernels. Mathematical Control and Related Fields, 2015, 5 (2) : 191-235. doi: 10.3934/mcrf.2015.5.191

2021 Impact Factor: 1.169

Metrics

  • PDF downloads (137)
  • HTML views (171)
  • Cited by (0)

Other articles
by authors

[Back to Top]