American Institute of Mathematical Sciences

Advanced
  • Previous Article
    Determining the waveguide conductivity in a hyperbolic equation from a single measurement on the lateral boundary
  • MCRF Home
  • This Issue
  • Next Article
    A sparse Markov chain approximation of LQ-type stochastic control problems
September  2016, 6(3): 391-406. doi: 10.3934/mcrf.2016008

On the convergence of the Sakawa-Shindo algorithm in stochastic control

J. Frédéric Bonnans 1, , Justina Gianatti 2, and  Francisco J. Silva 3,

1. 

INRIA-Saclay and Centre de Mathématiques Appliquées, Ecole Polytechnique and Laboratoire de Finance des Marchés d'Énergie, 91128 Palaiseau, France
2. 

CIFASIS - Centro Internacional Franco Argentino, de Ciencias de la Información y de Sistemas, CONICET - UNR - AMU, S2000EZP Rosario, Argentina
3. 

Institut de recherche XLIM-DMI, UMR-CNRS 7252, Faculté des sciences et techniques, Université de Limoges, 87060 Limoges, France

Received  May 2015 Revised  August 2015 Published  August 2016

We analyze an algorithm for solving stochastic control problems, based on Pontryagin's maximum principle, due to Sakawa and Shindo in the deterministic case and extended to the stochastic setting by Mazliak. We assume that either the volatility is an affine function of the state, or the dynamics are linear. We obtain a monotone decrease of the cost functions as well as, in the convex case, the fact that the sequence of controls is minimizing, and converges to an optimal solution if it is bounded. In a specific case we interpret the algorithm as the gradient plus projection method and obtain a linear convergence rate to the solution.
Keywords: Pontryagin's principle, first order algorithm., Stochastic control.
Mathematics Subject Classification: Primary: 93E20, 49M05; Secondary: 49K45, 93E25, 60H10, 60H3.
Citation: J. Frédéric Bonnans, Justina Gianatti, Francisco J. Silva. On the convergence of the Sakawa-Shindo algorithm in stochastic control. Mathematical Control & Related Fields, 2016, 6 (3) : 391-406. doi: 10.3934/mcrf.2016008
References:
[1]

J. Backhoff and F. J. Silva, Sensitivity results in stochastic optimal control: A Lagrangian perspective,, ESAIM: COCV, ().  doi: 10.1051/cocv/2015039.  Google Scholar

[2]

Lectures notes in Maths. Vol. 972, Springer-Verlag, Berlin, 1982.  Google Scholar

[3]

J. Franklin Inst., 315 (1983), 387-406, doi: 10.1016/0016-0032(83)90059-5.  Google Scholar

[4]

SIAM J. Control Optimization, 14 (1976), 419-444. doi: 10.1137/0314028.  Google Scholar

[5]

J. Math. Anal. Appl., 44 (1973), 384-404. doi: 10.1016/0022-247X(73)90066-8.  Google Scholar

[6]

SIAM Rev., 20 (1978), 62-78. doi: 10.1137/1020004.  Google Scholar

[7]

SIAM J. Control Optim., 24 (1986), 579-588. doi: 10.1137/0324034.  Google Scholar

[8]

Appl. Math. Optim., 65 (2012), 403-439. doi: 10.1007/s00245-012-9162-4.  Google Scholar

[9]

SIAM J. Control Optim., 33 (1995), 590-624. doi: 10.1137/S0363012992240722.  Google Scholar

[10]

Bull. Amer. Math. Soc., 70 (1964), 709-710. doi: 10.1090/S0002-9904-1964-11178-2.  Google Scholar

[11]

SIAM Rev., 23 (1981), 292-307. doi: 10.1137/1023062.  Google Scholar

[12]

SIAM J. Control, 10 (1972), 550-565. doi: 10.1137/0310041.  Google Scholar

[13]

J. Math. Anal. Appl., 11 (1965), 78-92. doi: 10.1016/0022-247X(65)90070-3.  Google Scholar

[14]

J. Math. Anal. Appl., 8 (1964), 287-302. doi: 10.1016/0022-247X(64)90070-8.  Google Scholar

[15]

Stochastic analysis and applications, 14 (1996), 513-533. doi: 10.1080/07362999608809455.  Google Scholar

[16]

Pure Appl. Math. Q., 3 (2007), 539-567. doi: 10.4310/PAMQ.2007.v3.n2.a7.  Google Scholar

[17]

SIAM J. Control Optim., 28 (1990), 966-979. doi: 10.1137/0328054.  Google Scholar

[18]

Gordon & Breach Science Publishers, New York, 1986, Reprint of the 1962 English translation.  Google Scholar

[19]

IEEE Trans. Automat. Control, 25 (1980), 1149-1153. doi: 10.1109/TAC.1980.1102517.  Google Scholar

[20]

Springer-Verlag, New York, 1999. doi: 10.1007/978-1-4612-1466-3.  Google Scholar

show all references

References:
[1]

J. Backhoff and F. J. Silva, Sensitivity results in stochastic optimal control: A Lagrangian perspective,, ESAIM: COCV, ().  doi: 10.1051/cocv/2015039.  Google Scholar

[2]

Lectures notes in Maths. Vol. 972, Springer-Verlag, Berlin, 1982.  Google Scholar

[3]

J. Franklin Inst., 315 (1983), 387-406, doi: 10.1016/0016-0032(83)90059-5.  Google Scholar

[4]

SIAM J. Control Optimization, 14 (1976), 419-444. doi: 10.1137/0314028.  Google Scholar

[5]

J. Math. Anal. Appl., 44 (1973), 384-404. doi: 10.1016/0022-247X(73)90066-8.  Google Scholar

[6]

SIAM Rev., 20 (1978), 62-78. doi: 10.1137/1020004.  Google Scholar

[7]

SIAM J. Control Optim., 24 (1986), 579-588. doi: 10.1137/0324034.  Google Scholar

[8]

Appl. Math. Optim., 65 (2012), 403-439. doi: 10.1007/s00245-012-9162-4.  Google Scholar

[9]

SIAM J. Control Optim., 33 (1995), 590-624. doi: 10.1137/S0363012992240722.  Google Scholar

[10]

Bull. Amer. Math. Soc., 70 (1964), 709-710. doi: 10.1090/S0002-9904-1964-11178-2.  Google Scholar

[11]

SIAM Rev., 23 (1981), 292-307. doi: 10.1137/1023062.  Google Scholar

[12]

SIAM J. Control, 10 (1972), 550-565. doi: 10.1137/0310041.  Google Scholar

[13]

J. Math. Anal. Appl., 11 (1965), 78-92. doi: 10.1016/0022-247X(65)90070-3.  Google Scholar

[14]

J. Math. Anal. Appl., 8 (1964), 287-302. doi: 10.1016/0022-247X(64)90070-8.  Google Scholar

[15]

Stochastic analysis and applications, 14 (1996), 513-533. doi: 10.1080/07362999608809455.  Google Scholar

[16]

Pure Appl. Math. Q., 3 (2007), 539-567. doi: 10.4310/PAMQ.2007.v3.n2.a7.  Google Scholar

[17]

SIAM J. Control Optim., 28 (1990), 966-979. doi: 10.1137/0328054.  Google Scholar

[18]

Gordon & Breach Science Publishers, New York, 1986, Reprint of the 1962 English translation.  Google Scholar

[19]

IEEE Trans. Automat. Control, 25 (1980), 1149-1153. doi: 10.1109/TAC.1980.1102517.  Google Scholar

[20]

Springer-Verlag, New York, 1999. doi: 10.1007/978-1-4612-1466-3.  Google Scholar

[1]

Shi'an Wang, N. U. Ahmed. Optimal control and stabilization of building maintenance units based on minimum principle. Journal of Industrial & Management Optimization, 2021, 17 (4) : 1713-1727. doi: 10.3934/jimo.2020041
[2]

Bin Pei, Yong Xu, Yuzhen Bai. Convergence of p-th mean in an averaging principle for stochastic partial differential equations driven by fractional Brownian motion. Discrete & Continuous Dynamical Systems - B, 2020, 25 (3) : 1141-1158. doi: 10.3934/dcdsb.2019213
[3]

Fabio Sperotto Bemfica, Marcelo Mendes Disconzi, Casey Rodriguez, Yuanzhen Shao. Local existence and uniqueness in Sobolev spaces for first-order conformal causal relativistic viscous hydrodynamics. Communications on Pure & Applied Analysis, , () : -. doi: 10.3934/cpaa.2021069
[4]

Diana Keller. Optimal control of a linear stochastic Schrödinger equation. Conference Publications, 2013, 2013 (special) : 437-446. doi: 10.3934/proc.2013.2013.437
[5]

Omer Gursoy, Kamal Adli Mehr, Nail Akar. Steady-state and first passage time distributions for waiting times in the $ MAP/M/s+G $ queueing model with generally distributed patience times. Journal of Industrial & Management Optimization, 2021  doi: 10.3934/jimo.2021078
[6]

Qi Lü, Xu Zhang. A concise introduction to control theory for stochastic partial differential equations. Mathematical Control & Related Fields, 2021  doi: 10.3934/mcrf.2021020
[7]

Li Chu, Bo Wang, Jie Zhang, Hong-Wei Zhang. Convergence analysis of a smoothing SAA method for a stochastic mathematical program with second-order cone complementarity constraints. Journal of Industrial & Management Optimization, 2021, 17 (4) : 1863-1886. doi: 10.3934/jimo.2020050
[8]

Seddigheh Banihashemi, Hossein Jafaria, Afshin Babaei. A novel collocation approach to solve a nonlinear stochastic differential equation of fractional order involving a constant delay. Discrete & Continuous Dynamical Systems - S, 2021  doi: 10.3934/dcdss.2021025
[9]

Shanjian Tang, Fu Zhang. Path-dependent optimal stochastic control and viscosity solution of associated Bellman equations. Discrete & Continuous Dynamical Systems, 2015, 35 (11) : 5521-5553. doi: 10.3934/dcds.2015.35.5521
[10]

Wensheng Yin, Jinde Cao, Guoqiang Zheng. Further results on stabilization of stochastic differential equations with delayed feedback control under $ G $-expectation framework. Discrete & Continuous Dynamical Systems - B, 2021  doi: 10.3934/dcdsb.2021072
[11]

Kehan Si, Zhenda Xu, Ka Fai Cedric Yiu, Xun Li. Open-loop solvability for mean-field stochastic linear quadratic optimal control problems of Markov regime-switching system. Journal of Industrial & Management Optimization, 2021  doi: 10.3934/jimo.2021074
[12]

Skyler Simmons. Stability of Broucke's isosceles orbit. Discrete & Continuous Dynamical Systems, 2021, 41 (8) : 3759-3779. doi: 10.3934/dcds.2021015
[13]

Meiqiao Ai, Zhimin Zhang, Wenguang Yu. First passage problems of refracted jump diffusion processes and their applications in valuing equity-linked death benefits. Journal of Industrial & Management Optimization, 2021  doi: 10.3934/jimo.2021039
[14]

Hao Li, Honglin Chen, Matt Haberland, Andrea L. Bertozzi, P. Jeffrey Brantingham. PDEs on graphs for semi-supervised learning applied to first-person activity recognition in body-worn video. Discrete & Continuous Dynamical Systems, 2021  doi: 10.3934/dcds.2021039
[15]

Muhammad Aslam Noor, Khalida Inayat Noor. Properties of higher order preinvex functions. Numerical Algebra, Control & Optimization, 2021, 11 (3) : 431-441. doi: 10.3934/naco.2020035
[16]

Demetres D. Kouvatsos, Jumma S. Alanazi, Kevin Smith. A unified ME algorithm for arbitrary open QNMs with mixed blocking mechanisms. Numerical Algebra, Control & Optimization, 2011, 1 (4) : 781-816. doi: 10.3934/naco.2011.1.781
[17]

Ashkan Ayough, Farbod Farhadi, Mostafa Zandieh, Parisa Rastkhadiv. Genetic algorithm for obstacle location-allocation problems with customer priorities. Journal of Industrial & Management Optimization, 2021, 17 (4) : 1753-1769. doi: 10.3934/jimo.2020044
[18]

Yves Dumont, Frederic Chiroleu. Vector control for the Chikungunya disease. Mathematical Biosciences & Engineering, 2010, 7 (2) : 313-345. doi: 10.3934/mbe.2010.7.313
[19]

Z. Reichstein and B. Youssin. Parusinski's "Key Lemma" via algebraic geometry. Electronic Research Announcements, 1999, 5: 136-145.

[20]

Ugo Bessi. Another point of view on Kusuoka's measure. Discrete & Continuous Dynamical Systems, 2021, 41 (7) : 3241-3271. doi: 10.3934/dcds.2020404

2019 Impact Factor: 0.857

Tools

Metrics

  • PDF downloads (68)
  • HTML views (0)
  • Cited by (0)

Other articles
by authors

[Back to Top]

Copyright © 2021 American Institute of Mathematical Sciences