On the convergence of the Sakawa-Shindo  algorithm  in stochastic control

J. Frédéric  Bonnans; Justina  Gianatti; Francisco J.  Silva

doi:10.3934/mcrf.2016008

Article Contents

2016, Volume 6, Issue 3: 391-406. Doi: 10.3934/mcrf.2016008

This issue Previous Article A sparse Markov chain approximation of LQ-type stochastic control problems Next Article Determining the waveguide conductivity in a hyperbolic equation from a single measurement on the lateral boundary

On the convergence of the Sakawa-Shindo algorithm in stochastic control

1.
INRIA-Saclay and Centre de Mathématiques Appliquées, Ecole Polytechnique and Laboratoire de Finance des Marchés d'Énergie, 91128 Palaiseau
2.
CIFASIS - Centro Internacional Franco Argentino, de Ciencias de la Información y de Sistemas, CONICET - UNR - AMU, S2000EZP Rosario
3.
Institut de recherche XLIM-DMI, UMR-CNRS 7252, Faculté des sciences et techniques, Université de Limoges, 87060 Limoges

Received: April 30, 2015

Revised: July 31, 2015

Published: August 2016

Abstract Related Papers Cited by

Abstract

We analyze an algorithm for solving stochastic control problems, based on Pontryagin's maximum principle, due to Sakawa and Shindo in the deterministic case and extended to the stochastic setting by Mazliak. We assume that either the volatility is an affine function of the state, or the dynamics are linear. We obtain a monotone decrease of the cost functions as well as, in the convex case, the fact that the sequence of controls is minimizing, and converges to an optimal solution if it is bounded. In a specific case we interpret the algorithm as the gradient plus projection method and obtain a linear convergence rate to the solution.

Keywords:

Mathematics Subject Classification: Primary: 93E20, 49M05; Secondary: 49K45, 93E25, 60H10, 60H35.

Citation:

Related Papers

Cited by

References

[1]	J. Backhoff and F. J. Silva, Sensitivity results in stochastic optimal control: A Lagrangian perspective, ESAIM: COCV, to appear. doi: 10.1051/cocv/2015039.
[2]	A. Bensoussan, Lectures on Stochastic Control, Lectures notes in Maths. Vol. 972, Springer-Verlag, Berlin, 1982.
[3]	A. Bensoussan, Stochastic maximum principle for distributed parameter systems, J. Franklin Inst., 315 (1983), 387-406,doi: 10.1016/0016-0032(83)90059-5.
[4]	J.-M. Bismut, Linear quadratic optimal stochastic control with random coefficients, SIAM J. Control Optimization, 14 (1976), 419-444.doi: 10.1137/0314028.
[5]	J.-M. Bismut, Conjugate convex functions in optimal stochastic control, J. Math. Anal. Appl., 44 (1973), 384-404.doi: 10.1016/0022-247X(73)90066-8.
[6]	J.-M. Bismut, An introductory approach to duality in optimal stochastic control, SIAM Rev., 20 (1978), 62-78.doi: 10.1137/1020004.
[7]	J. F. Bonnans, On an algorithm for optimal control using Pontryagin's maximum principle, SIAM J. Control Optim., 24 (1986), 579-588.doi: 10.1137/0324034.
[8]	J. F. Bonnans and F. J. Silva, First and second order necessary conditions for stochastic optimal control problems, Appl. Math. Optim., 65 (2012), 403-439.doi: 10.1007/s00245-012-9162-4.
[9]	A. Cadenillas and I. Karatzas, The stochastic maximum principle for linear convex optimal control with random coefficients, SIAM J. Control Optim., 33 (1995), 590-624.doi: 10.1137/S0363012992240722.
[10]	A. Goldstein, Convex programming in Hilbert space, Bull. Amer. Math. Soc., 70 (1964), 709-710.doi: 10.1090/S0002-9904-1964-11178-2.
[11]	U. G. Haussmann, Some examples of optimal stochastic controls or: The stochastic maximum principle at work, SIAM Rev., 23 (1981), 292-307.doi: 10.1137/1023062.
[12]	H. J. Kushner, Necessary conditions for continuous parameter stochastic optimization problems, SIAM J. Control, 10 (1972), 550-565.doi: 10.1137/0310041.
[13]	H. J. Kushner, On the stochastic maximum principle: Fixed time of control, J. Math. Anal. Appl., 11 (1965), 78-92.doi: 10.1016/0022-247X(65)90070-3.
[14]	H. J. Kushner and F. C. Schweppe, A maximum principle for stochastic control systems, J. Math. Anal. Appl., 8 (1964), 287-302.doi: 10.1016/0022-247X(64)90070-8.
[15]	L. Mazliak, An algorithm for solving a stochastic control problem, Stochastic analysis and applications, 14 (1996), 513-533.doi: 10.1080/07362999608809455.
[16]	L. Mou and J. Yong, A variational formula for stochastic controls and some applications, Pure Appl. Math. Q., 3 (2007), 539-567.doi: 10.4310/PAMQ.2007.v3.n2.a7.
[17]	S. G. Peng, A general stochastic maximum principle for optimal control problems, SIAM J. Control Optim., 28 (1990), 966-979.doi: 10.1137/0328054.
[18]	L. Pontryagin, V. Boltyanskiĭ, R. Gamkrelidze and E. Mishchenko, The Mathematical Theory of Optimal Processes, Gordon & Breach Science Publishers, New York, 1986, Reprint of the 1962 English translation.
[19]	Y. Sakawa and Y. Shindo, On global convergence of an algorithm for optimal control, IEEE Trans. Automat. Control, 25 (1980), 1149-1153.doi: 10.1109/TAC.1980.1102517.
[20]	J. Yong and X. Zhou, Stochastic Controls, Hamiltonian Systems and HJB Equations, Springer-Verlag, New York, 1999.doi: 10.1007/978-1-4612-1466-3.