The stochastic linear quadratic optimal control problem in Hilbert spaces: A polynomial chaos approach

Tijana  Levajković; Hermann  Mena; Amjad  Tuffaha

doi:10.3934/eect.2016.5.105

Article Contents

2016, Volume 5, Issue 1: 105-134. Doi: 10.3934/eect.2016.5.105

This issue Previous Article Elliptic and parabolic equations with fractional diffusion and dynamic boundary conditions Next Article The energy conservation for weak solutions to the relativistic Nordström-Vlasov system

The stochastic linear quadratic optimal control problem in Hilbert spaces: A polynomial chaos approach

1.
Department of Mathematics, Faculty of Mathematics, Computer Science and Physics, University of Innsbruck, Innsbruck, A 6020
2.
The American University of Sharjah, Sharjah

Received: October 31, 2015

Revised: December 31, 2015

Published: February 2016

Abstract Related Papers Cited by

Abstract

We consider the stochastic linear quadratic optimal control problem for state equations of the Itô-Skorokhod type, where the dynamics are driven by strongly continuous semigroup. We provide a numerical framework for solving the control problem using a polynomial chaos expansion approach in white noise setting. After applying polynomial chaos expansion to the state equation, we obtain a system of infinitely many deterministic partial differential equations in terms of the coefficients of the state and the control variables. We set up a control problem for each equation, which results in a set of deterministic linear quadratic regulator problems. Solving these control problems, we find optimal coefficients for the state and the control. We prove the optimality of the solution expressed in terms of the expansion of these coefficients compared to a direct approach. Moreover, we apply our result to a fully stochastic problem, in which the state, control and observation operators can be random, and we also consider an extension to state equations with memory noise.

Keywords:

Stochastic linear quadratic optimal control problem,
white noise analysis,
polynomial chaos expansion method,
Itô-Skorokhod integral,
Riccati equations.

Mathematics Subject Classification: Primary: 34H05, 49J55; Secondary: 15A24, 60H40, 60H30, 93E20.

Citation:

Related Papers

Cited by

References

[1]	E. Arias, V. Hernández, J. Ibanes and J. Peinado, A family of BDF algorithms for solving differential matrix Riccati equations using adaptive techniques, Procedia Computer Science, 1 (2010), 2569-2577.
[2]	G. Avalos and I. Lasiecka, Differential Riccati equation for the active control of a problem in structural acoustics, J. Optim. Theory Appl., 91 (1996), 695-728.doi: 10.1007/BF02190128.
[3]	H. T. Banks, R. J. Silcox and R. C. Smith, The modeling and control of acoustic/structure interaction problems via piezoceramic actuators: 2-d numerical examples, ASME J. Vibration Acoustics, 116 (1994), 386-396.doi: 10.1115/1.2930440.
[4]	H. T. Banks, R. C. Smith and Y. Wang, The modeling of piezoceramic patch interactions with shells, plates and beams, Quart. Appl. Math, 53 (1995), 353-381.
[5]	P. Benner, P. Ezzatti, H. Mena, E. S. Quintana-Ortí and A. Remón, Solving matrix equations on multi-core and many-core architectures, Algorithms, 6 (2013), 857-870.doi: 10.3390/a6040857.
[6]	P. Benner and H. Mena, Numerical solution of the infinite-dimensional LQR-problem and the associated differential Riccati equations, MPI Magdeburg Preprint MPIMD/12-13, 2012, Available from: http://www2.mpi-magdeburg.mpg.de/preprints/2012/MPIMD12-13.pdf.
[7]	P. Benner and H. Mena, Rosenbrock methods for solving differential Riccati equations, IEEE Transactions on Automatic Control, 58 (2013), 2950-2956.doi: 10.1109/TAC.2013.2258495.
[8]	F. E. Benth and T. G. Theting, Some regularity results for the stochastic pressure equation of Wick-type, Stochastic Analysis and Applications, 20 (2002), 1191-1223.doi: 10.1081/SAP-120015830.
[9]	J.-M. Bismut, Linear quadratic optimal stochastic control with random coefficients, SIAM J. Control. Optim., 14 (1976), 419-444.doi: 10.1137/0314028.
[10]	J.-M. Bismut, Contrôle des systmes linéaires quadratiques: Applications de l'intégrale stochastique, in Séminaire de Probabilités XII, Lecture Notes in Math., 649 (1978), Springer, Berlin, 180-264.
[11]	K. R. Dahl, S.-E. A. Mohammed, B. Øksendal and E. E. Røse, Optimal control of systems with noisy memory and BSDEs with Malliavin derivatives, preprint, arXiv:1403.4034v3.
[12]	G. Da Prato, Direct solution of a Riccati equation arising in stochastic control theory, Appl. Math. Optim., 11 (1984), 191-208.doi: 10.1007/BF01442178.
[13]	G. Da Prato and J. Zabczyk, Stochastic Equations in Infinite Dimensions, $2^{nd}$ edition, Encyclopedia of Mathematics and its Applications, 152. Cambridge University Press, Cambridge, 2014.doi: 978-1-107-05584-1.
[14]	F. Flandoli, Direct solution of a Riccati equation arising in a stochastic control problem with control and observation on the boundary, Appl. Math. Optim, 14 (1986), 107-129.doi: 10.1007/BF01442231.
[15]	H. Fleming and H. M. Soner, Controlled Markov Processes and Viscosity Solutions, $2^{nd}$ edition, Stochastic Modelling and Applied Probability 25, Springer, New York, 2006.
[16]	J. Fisher and R. Bhattacharya, On stochastic LQR design and polynomial chaos, in American Control Conference, Seattle, WA, (2008), 95-100.doi: 10.1109/ACC.2008.4586473.
[17]	J. Fisher and R. Bhattacharya, Stability analysis of stochastic systems using polynomial chaos, In American Control Conference, Seattle, WA, (2008), 4250-4255.doi: 10.1109/ACC.2008.4587161.
[18]	R. Ghanem and P. D. Spanos, Polynomial chaos in stochastic finite elements, Journal of Applied Mechanics, 57 (1990), 197-202.doi: 10.1115/1.2888303.
[19]	M. Grothaus, Y. G. Kondratiev and G. F. Us, Wick calculus for regular generalized stochastic functionals, Random Oper. Stochastic Equations, 7 (1999), 263-290.doi: 10.1515/rose.1999.7.3.263.
[20]	G. Guatteri and G. Tessitore, On the backward stochastic Riccati equation in infinite dimensions, SIAM J. Control Optim., 44 (2005), 159-194.doi: 10.1137/S0363012903425507.
[21]	G. Guatteri and G. Tessitore, Backward stochastic Riccati equations and infinite horizon L-Q optimal control with infinite dimensional state space and random coefficients, Appl. Math. Optim., 57 (2008), 207-235.doi: 10.1007/s00245-007-9020-y.
[22]	C. Hafizoglu, Linear Quadratic Boundary{/Point Control of Stochastic Partial Differential Equation Systems with Unbounded Coefficients}, Ph.D thesis, University of Virginia, 2006.
[23]	C. Hafizoglu, I. Lasiecka, T. Levajković, H. Mena and A. Tuffaha, The stochastic linear quadratic control problem with singular estimates, preprint, 2015.
[24]	T. Hida, H.-H. Kuo, J. Potthoff and L. Streit, White Noise. An Infinite-Dimensional Calculus, Mathematics and its Applications, 253, Kluwer Academic Publishers Group, Dordrecht, 1993.doi: 10.1007/978-94-017-3680-0.
[25]	H. Holden, B. Øksendal, J. Ubøe and T. Zhang, Stochastic Partial Differential Equations. A modeling, White Noise Functional Approach, $2^{nd}$ edition, Springer, 2010.doi: 10.1007/978-0-387-89488-1.
[26]	F. S. Hover and M. S. Triantafyllou, Application of polynomial chaos in stability and control, Automatica, 42 (2006), 789-795.doi: 10.1016/j.automatica.2006.01.010.
[27]	A. Ichikawa, Dynamic programming approach to stochastic evolution equations, SIAM J. Control. Optim., 17 (1979), 152-174.doi: 10.1137/0317012.
[28]	M. Kohlmann and S. Tang, New developments in backward stochastic Riccati equations and their applications, In: Mathematical Finance (Konstanz 2000), Trends Math. Birkhäuser, Basel, 194-214, 2001.
[29]	M. Kohlmann and S. Tang, Global adapted solution of one-dimensional backward stochastic Riccati equations, with application to the mean-variance hedging, Stoch. Process. Appl., 97 (2002), 255-288.doi: 10.1016/S0304-4149(01)00133-8.
[30]	M. Kohlmann and S. Tang, Multidimensional backward stochastic Riccati equations and applications, SIAM J. Control. Optim., 41 (2003), 1696-1721.doi: 10.1137/S0363012900378760.
[31]	M. Kohlmann and X. Y. Zhou, Relationship between backward stochastic differential equations and stochastic controls: A linear-quadratic approach, SIAM J. Control. Optim., 38 (2000), 1392-1407.doi: 10.1137/S036301299834973X.
[32]	H. J. Kushner, Optimal stochastic control, IRE Trans. Auto. Control, 7 (1962), 120-122.doi: 10.1109/TAC.1962.1105490.
[33]	N. Lang, H. Mena and J. Saak, On the benefits of the LDL factorization for large-scale differential matrix equation solvers, Linear Algebra and its Applications, 480 (2015), 44-71.doi: 10.1016/j.laa.2015.04.006.
[34]	I. Lasiecka and R. Triggiani, Control Theory for Partial Differential Equations: Continuous and Approximation Theories I. Abstract Parabolic Systems, Encyclopedia of Mathematics and its Applications 74, Cambridge University Press, 2000.
[35]	I. Lasiecka and R. Triggiani, Control Theory for Partial Differential Equations: Continuous and Approximation Theories II. Abstract Hyperbolic-Like Systems over a Finite Time Horizon, Encyclopedia of Mathematics and its Applications 75, Cambridge University Press, 2000.doi: 10.1017/CBO9780511574801.002.
[36]	I. Lasiecka and R. Triggiani, Optimal control and differential Riccati equations under singular estimates for $e^{At}B$ in the absence of analyticity, Advances in Dynamics and Control, Nonlinear Syst. Aviat. Aerosp. Aeronaut. Astronaut., 2, CRC Press (2004), 270-307.
[37]	C. Lebdzek and R. Triggiani, Optimal regularity and optimal control of a thermoelastic structural acoustic model with point control and clamped boundary conditions, Control Cybernet, 38 (2009), 1461-1499.
[38]	T. Levajković and H. Mena, On deterministic and stochastic linear quadratic control problem, Current Trends in Analysis and Its Applications, Trends in Mathematics, Research Perspectives, Springer International Publishing Switzerland (2015), 315-322.
[39]	T. Levajković, H. Mena and A. Tuffaha, A Numerical approximation framework for the stochastic linear quadratic regulator on Hilbert spaces, Appl. Math. Optim., (2016).doi: 10.1007/s00245-016-9339-3.
[40]	T. Levajković, S. Pilipović and D. Seleši, The stochastic Dirichlet problem driven by the Ornstein-Uhlenbeck operator$:$ Approach by the Fredholm alternative for chaos expansions, Stoch. Anal. Appl., 29 (2011), 317-331.doi: 10.1080/07362994.2011.548998.
[41]	T. Levajković and D. Seleši, Chaos expansion methods for stochastic differential equations involving the Malliavin derivative Part I, Publ. Inst. Math. (Beograd) (N.S.), 90 (2011), 65-84.doi: 10.2298/PIM1104065L.
[42]	T. Levajković, S. Pilipović and D. Seleši, Fundamental equations with higher order Malliavin operators, Stochastics: An International Journal of Probability and Stochastic Processes, 88 (2016), 106-127.doi: 10.1080/17442508.2015.1036434.
[43]	T. Levajković, S. Pilipović and D. Seleši, Chaos expansion methods in Malliavin calculus: A survey of recent results, Novi Sad J. Math., 45 (2015), 45-103.
[44]	T. Levajković, S. Pilipović, D. Seleši and M. Žigić, Stochastic evolution equations with multiplicative noise, Electronic Journal of Probability, 20 (2015), 23pp.doi: 10.1214/EJP.v20-3696.
[45]	S. Lototsky and B. Rozovskii, Stochastic differential equations: A Wiener chaos approach, From stochastic calculus to mathematical finance, (eds. Yu. Kabanov et al.), Springer Berlin (2006), 433-506.doi: 10.1007/978-3-540-30788-4_23.
[46]	E. A. Kalpinelli, N. E. Frangos and A. N. Yannacopoulos, Numerical methods for hyperbolic SPDEs: A Wiener chaos approach, Stoch. Partial Differ. Equ. Anal. Comput., 1 (2013), 606-633.doi: 10.1007/s40072-013-0019-x.
[47]	H. Matthies, Stochastic finite elements: Computational approaches to stochastic partial differential equations, Z. Angew. Math. Mech., 88 (2008), 849-873.doi: 10.1002/zamm.200800095.
[48]	R. Mikulevicius and B. Rozovskii, On unbiased stochastic Navier-Stokes equations, Probab. Theory Related Fields, 154 (2012), 787-834.doi: 10.1007/s00440-011-0384-1.
[49]	A. Monti, F. Ponci and T. Lovett, A polynomial chaos theory approach to the control design of a power converter, In Power Electronics Specialists Conference, PESC 04, IEEE 35th Annual, (6) (2004), Aachen, Germany, 4809-4813.doi: 10.1109/PESC.2004.1354850.
[50]	D. Nualart, The Malliavin Calculus and Related Topics, $2^{nd}$ edition, Probability and its Applications, Springer-Verlag, Berlin, 2006.
[51]	A. Pazy, Semigroups of Linear Operators and Applications to Partial Differential Equations, Applied Mathematical Sciences 44, Springer-Verlag, New York Inc, 1983.doi: 10.1007/978-1-4612-5561-1.
[52]	S. Peng, Stochastic Hamilton-Jacobi-Bellman equations, SIAM J. Control. Optim., 30 (1992), 284-304.doi: 10.1137/0330018.
[53]	S. Peng, Open problems on backward stochastic differential equations, in Control of Distributed Parameter and Stochastic Systems, (Hangzhou 1998), Kluwer Academic, Boston, 1999, 265-273.
[54]	S. Pilipović and D. Seleši, Expansion theorems for generalized random processes, Wick products and applications to stochastic differential equations, Infin. Dimens. Anal. Quantum Probab. Relat. Top., 10 (2007), 79-110.doi: 10.1142/S0219025707002634.
[55]	A. Sandu, C. Sandu, B. J. Chan and M. Ahmadian, Control mechanical systems using a parameterized spectral decomposition approach, in Proceedings of the IMECE04, ASME International Sixth Annual Symposium on Advanced Vehicle Technologies, Anaheim, CA, November 2004.
[56]	W. Schoutens, Stochastic Processes and Orthogonal Polynomials, Lecture Notes in Statistics 146, Springer Verlag, 2000.doi: 10.1007/978-1-4612-1170-9.
[57]	B. A. Templeton, A Polynomial Chaos Approach to Control Design, Ph.D Thesis, Virginia Polytechnic University, 2009.
[58]	D. Venturi, X. Wan, R. Mikulevicius, B. L. Rozovskii and G. E. Karniadakis, Wick - Malliavin approximation to nonlinear stochastic partial differential equations: Analysis and simulations, Proc. R. Soc. Lond. Ser. A Math. Phys. Eng. Sci., 469 (2013), 20130001, 20pp.doi: 10.1098/rspa.2013.0001.
[59]	N. Wiener, The homogeneous chaos, American Journal of Mathematics, 60 (1938), 897-936.doi: 10.2307/2371268.
[60]	W. M. Wonham, On the separation theorem of stochastic control, SIAM J. Control, 6 (1968), 312-326.doi: 10.1137/0306023.
[61]	W. M. Wonham, On a matrix Riccati equation of stochastic control, SIAM J. Control, 6 (1968), 681-697.doi: 10.1137/0306044.
[62]	D. Xiu and G. Karniadakis, The Wiener-Askey polynomial chaos for stochastic differential equations, SIAM J. Sci. Comput., 24 (2002), 619-644.doi: 10.1137/S1064827501387826.
[63]	J. Yong and X. Y. Zhou, Stochastic Controls - Hamiltonian Systems and HJB Equations, Applications of Mathematics, Stochastic Modelling and Applied Probability 43, Springer, New York, 1999.doi: 10.1007/978-1-4612-1466-3.