doi: 10.3934/mcrf.2020053

Strict dissipativity analysis for classes of optimal control problems involving probability density functions

1. 

Chair of Serious Games, University of Bayreuth, 95440 Bayreuth, Germany

2. 

Chair of Applied Mathematics, University of Bayreuth, 95440 Bayreuth, Germany

* Corresponding author: Arthur Fleig

Received  July 2019 Revised  March 2020 Published  December 2020

Fund Project: The first author was supported by DFG grant GR 1569/15-1

Motivated by the stability and performance analysis of model predictive control schemes, we investigate strict dissipativity for a class of optimal control problems involving probability density functions. The dynamics are governed by a Fokker-Planck partial differential equation. However, for the particular classes under investigation involving linear dynamics, linear feedback laws, and Gaussian probability density functions, we are able to significantly simplify these dynamics. This enables us to perform an in-depth analysis of strict dissipativity for different cost functions.

Citation: Arthur Fleig, Lars Grüne. Strict dissipativity analysis for classes of optimal control problems involving probability density functions. Mathematical Control & Related Fields, doi: 10.3934/mcrf.2020053
References:
[1]

Y. AchdouF. Camilli and I. Capuzzo-Dolcetta, Mean field games: Numerical methods for the planning problem, SIAM J. Control Optim., 50 (2012), 77-109.  doi: 10.1137/100790069.  Google Scholar

[2]

D. AngeliR. Amrit and J. B. Rawlings, On average performance and stability of economic model predictive control, IEEE Trans. Autom. Control, 57 (2012), 1615-1626.  doi: 10.1109/TAC.2011.2179349.  Google Scholar

[3]

M. Annunziato and A. Borzì, Optimal control of probability density functions of stochastic processes, Math. Model. Anal., 15 (2010), 393-407.  doi: 10.3846/1392-6292.2010.15.393-407.  Google Scholar

[4]

M. Annunziato and A. Borzì, A Fokker-Planck control framework for multidimensional stochastic processes, J. Comput. Appl. Math., 237 (2013), 487-507.  doi: 10.1016/j.cam.2012.06.019.  Google Scholar

[5]

J.-D. Benamou and G. Carlier, Augmented Lagrangian methods for transport optimization, mean field games and degenerate elliptic equations, J. Optim. Theory Appl., 167 (2015), 1-26.  doi: 10.1007/s10957-015-0725-9.  Google Scholar

[6]

M. BonginiM. FornasierF. Rossi and F. Solombrino, Mean-field Pontryagin maximum principle, J. Optim. Theory Appl., 175 (2017), 1-38.  doi: 10.1007/s10957-017-1149-5.  Google Scholar

[7]

T. BreitenK. Kunisch and L. Pfeiffer, Control strategies for the Fokker-Planck equation, ESAIM: COCV, 24 (2018), 741-763.  doi: 10.1051/cocv/2017046.  Google Scholar

[8]

T. Breiten and L. Pfeiffer, On the turnpike property and the receding-horizon method for linear-quadratic optimal control problems, SIAM Journal on Control and Optimization, 58 (2020), 1077-1102.  doi: 10.1137/18M1225811.  Google Scholar

[9]

T. DammL. GrüneM. Stieler and K. Worthmann, An exponential turnpike theorem for dissipative discrete time optimal control problems, SIAM J. Control Optim., 52 (2014), 1935-1957.  doi: 10.1137/120888934.  Google Scholar

[10]

M. DiehlR. Amrit and J. B. Rawlings, A Lyapunov function for economic optimizing model predictive control, IEEE Trans. Autom. Control, 56 (2011), 703-707.  doi: 10.1109/TAC.2010.2101291.  Google Scholar

[11]

T. Faulwasser, L. Grüne and M. A. Müller, Economic nonlinear model predictive control, Foundations and Trends® in Systems and Control, 5 (2018), 1–98. Google Scholar

[12]

A. Fleig and L. Grüne, Estimates on the minimal stabilizing horizon length in model predictive control for the Fokker-Planck equation, IFAC-PapersOnLine, 49 (2016), 260-265.  doi: 10.1016/j.ifacol.2016.07.451.  Google Scholar

[13]

A. Fleig and L. Grüne, $L^2$-tracking of Gaussian distributions via model predictive control for the Fokker-Planck equation, Vietnam J. Math., 46 (2018), 915-948.  doi: 10.1007/s10013-018-0309-8.  Google Scholar

[14]

A. Fleig and L. Grüne, On dissipativity of the Fokker-Planck equation for the OrnsteinUhlenbeck process, in IFAC-PapersOnLine, 3rd IFAC Workshop on Control of Systems Governed by Partial Differential Equations CPDE 2019, 52 (2019), 13-18,  Google Scholar

[15]

C. R. Givens and R. M. Shortt, A class of wasserstein metrics for probability distributions, Michigan Math. J., 31 (1984), 231-240.  doi: 10.1307/mmj/1029003026.  Google Scholar

[16]

L. Grüne, Economic receding horizon control without terminal constraints, Automatica, 49 (2013), 725-734.  doi: 10.1016/j.automatica.2012.12.003.  Google Scholar

[17]

L. Grüne and R. Guglielmi, Turnpike properties and strict dissipativity for discrete time linear quadratic optimal control problems, SIAM J. Cont. Optim., 56 (2018), 1282-1302.  doi: 10.1137/17M112350X.  Google Scholar

[18]

L. Grüne and M. A. Müller, On the relation between strict dissipativity and the turnpike property, Syst. Contr. Lett., 90 (2016), 45-53.  doi: 10.1016/j.sysconle.2016.01.003.  Google Scholar

[19]

L. Grüne and J. Pannek, Nonlinear Model Predictive Control, Theory and Algorithms, 2nd edition, Springer, London, 2017. doi: 10.1007/978-3-319-46024-6.  Google Scholar

[20]

L. GrüneM. Schaller and A. Schiela, Sensitivity analysis of optimal control for a class of parabolic PDEs motivated by model predictive control, SIAM J. Control Optim., 57 (2019), 2753-2774.  doi: 10.1137/18M1223083.  Google Scholar

[21]

L. Grüne and M. Stieler, Asymptotic stability and transient optimality of economic MPC without terminal conditions, J. Proc. Control, 24 (2014), 1187-1196.   Google Scholar

[22]

L. GrüneM. Schaller and A. Schiela, Exponential sensitivity and turnpike analysis for linear quadratic optimal control of general evolution equations, J. Differ. Equ., 268 (2020), 7311-7341.  doi: 10.1016/j.jde.2019.11.064.  Google Scholar

[23]

W. Hahn, Stability of Motion, Springer, 1967.  Google Scholar

[24]

A. Porretta and E. Zuazua, Long time versus steady state optimal control, SIAM J. Control Optim., 51 (2013), 4242-4273.  doi: 10.1137/130907239.  Google Scholar

[25]

S. Primak, V. Kontorovich and V. Lyandres, Stochastic Methods and Their Applications to Communications, John Wiley & Sons, Inc., Hoboken, NJ, 2004.  Google Scholar

[26]

P. E. Protter, Stochastic Integration and Differential Equations, vol. 21 of Stochastic Modelling and Applied Probability, Springer-Verlag, Berlin, 2005. doi: 10.1007/978-3-662-10061-5.  Google Scholar

[27]

J. B. RawlingsD. BonnéJ. B. JørgensenA. N. Venkat and S. B. Jørgensen, Unreachable setpoints in model predictive control, IEEE Transactions on Automatic Control, 53 (2008), 2209-2215.  doi: 10.1109/TAC.2008.928125.  Google Scholar

[28]

J. B. Rawlings, D. Q. Mayne and M. M. Diehl, Model Predictive Control: Theory and Design, 2nd edition, Nob Hill Publishing, 2017. Google Scholar

[29]

H. Risken, The Fokker-Planck Equation, vol. 18 of Springer Series in Synergetics, 2nd edition, Springer-Verlag, Berlin, 1989. doi: 10.1007/978-3-642-61544-3.  Google Scholar

[30]

S. RoyM. Annunziato and A. Borzì, A Fokker-Planck feedback control-constrained approach for modelling crowd motion, J. Comput. Theor. Transp., 45 (2016), 442-458.  doi: 10.1080/23324309.2016.1189435.  Google Scholar

[31]

E. Trélat and E. Zuazua, The turnpike property in finite-dimensional nonlinear optimal control, J. Differential Equations, 258 (2015), 81-114.  doi: 10.1016/j.jde.2014.09.005.  Google Scholar

[32]

E. TrélatC. Zhang and E. Zuazua, Steady-state and periodic exponential turnpike property for optimal control problems in Hilbert spaces, SIAM J. Control Optim., 56 (2018), 1222-1252.  doi: 10.1137/16M1097638.  Google Scholar

[33]

F. Tröltzsch, Optimal Control of Partial Differential Equations, vol. 112 of Graduate Studies in Mathematics, American Mathematical Society, Providence, RI, 2010. doi: 10.1090/gsm/112.  Google Scholar

[34]

J. C. Willems, Dissipative dynamical systems. I. General theory, Arch. Rational Mech. Anal., 45 (1972), 321-351.  doi: 10.1007/BF00276493.  Google Scholar

show all references

References:
[1]

Y. AchdouF. Camilli and I. Capuzzo-Dolcetta, Mean field games: Numerical methods for the planning problem, SIAM J. Control Optim., 50 (2012), 77-109.  doi: 10.1137/100790069.  Google Scholar

[2]

D. AngeliR. Amrit and J. B. Rawlings, On average performance and stability of economic model predictive control, IEEE Trans. Autom. Control, 57 (2012), 1615-1626.  doi: 10.1109/TAC.2011.2179349.  Google Scholar

[3]

M. Annunziato and A. Borzì, Optimal control of probability density functions of stochastic processes, Math. Model. Anal., 15 (2010), 393-407.  doi: 10.3846/1392-6292.2010.15.393-407.  Google Scholar

[4]

M. Annunziato and A. Borzì, A Fokker-Planck control framework for multidimensional stochastic processes, J. Comput. Appl. Math., 237 (2013), 487-507.  doi: 10.1016/j.cam.2012.06.019.  Google Scholar

[5]

J.-D. Benamou and G. Carlier, Augmented Lagrangian methods for transport optimization, mean field games and degenerate elliptic equations, J. Optim. Theory Appl., 167 (2015), 1-26.  doi: 10.1007/s10957-015-0725-9.  Google Scholar

[6]

M. BonginiM. FornasierF. Rossi and F. Solombrino, Mean-field Pontryagin maximum principle, J. Optim. Theory Appl., 175 (2017), 1-38.  doi: 10.1007/s10957-017-1149-5.  Google Scholar

[7]

T. BreitenK. Kunisch and L. Pfeiffer, Control strategies for the Fokker-Planck equation, ESAIM: COCV, 24 (2018), 741-763.  doi: 10.1051/cocv/2017046.  Google Scholar

[8]

T. Breiten and L. Pfeiffer, On the turnpike property and the receding-horizon method for linear-quadratic optimal control problems, SIAM Journal on Control and Optimization, 58 (2020), 1077-1102.  doi: 10.1137/18M1225811.  Google Scholar

[9]

T. DammL. GrüneM. Stieler and K. Worthmann, An exponential turnpike theorem for dissipative discrete time optimal control problems, SIAM J. Control Optim., 52 (2014), 1935-1957.  doi: 10.1137/120888934.  Google Scholar

[10]

M. DiehlR. Amrit and J. B. Rawlings, A Lyapunov function for economic optimizing model predictive control, IEEE Trans. Autom. Control, 56 (2011), 703-707.  doi: 10.1109/TAC.2010.2101291.  Google Scholar

[11]

T. Faulwasser, L. Grüne and M. A. Müller, Economic nonlinear model predictive control, Foundations and Trends® in Systems and Control, 5 (2018), 1–98. Google Scholar

[12]

A. Fleig and L. Grüne, Estimates on the minimal stabilizing horizon length in model predictive control for the Fokker-Planck equation, IFAC-PapersOnLine, 49 (2016), 260-265.  doi: 10.1016/j.ifacol.2016.07.451.  Google Scholar

[13]

A. Fleig and L. Grüne, $L^2$-tracking of Gaussian distributions via model predictive control for the Fokker-Planck equation, Vietnam J. Math., 46 (2018), 915-948.  doi: 10.1007/s10013-018-0309-8.  Google Scholar

[14]

A. Fleig and L. Grüne, On dissipativity of the Fokker-Planck equation for the OrnsteinUhlenbeck process, in IFAC-PapersOnLine, 3rd IFAC Workshop on Control of Systems Governed by Partial Differential Equations CPDE 2019, 52 (2019), 13-18,  Google Scholar

[15]

C. R. Givens and R. M. Shortt, A class of wasserstein metrics for probability distributions, Michigan Math. J., 31 (1984), 231-240.  doi: 10.1307/mmj/1029003026.  Google Scholar

[16]

L. Grüne, Economic receding horizon control without terminal constraints, Automatica, 49 (2013), 725-734.  doi: 10.1016/j.automatica.2012.12.003.  Google Scholar

[17]

L. Grüne and R. Guglielmi, Turnpike properties and strict dissipativity for discrete time linear quadratic optimal control problems, SIAM J. Cont. Optim., 56 (2018), 1282-1302.  doi: 10.1137/17M112350X.  Google Scholar

[18]

L. Grüne and M. A. Müller, On the relation between strict dissipativity and the turnpike property, Syst. Contr. Lett., 90 (2016), 45-53.  doi: 10.1016/j.sysconle.2016.01.003.  Google Scholar

[19]

L. Grüne and J. Pannek, Nonlinear Model Predictive Control, Theory and Algorithms, 2nd edition, Springer, London, 2017. doi: 10.1007/978-3-319-46024-6.  Google Scholar

[20]

L. GrüneM. Schaller and A. Schiela, Sensitivity analysis of optimal control for a class of parabolic PDEs motivated by model predictive control, SIAM J. Control Optim., 57 (2019), 2753-2774.  doi: 10.1137/18M1223083.  Google Scholar

[21]

L. Grüne and M. Stieler, Asymptotic stability and transient optimality of economic MPC without terminal conditions, J. Proc. Control, 24 (2014), 1187-1196.   Google Scholar

[22]

L. GrüneM. Schaller and A. Schiela, Exponential sensitivity and turnpike analysis for linear quadratic optimal control of general evolution equations, J. Differ. Equ., 268 (2020), 7311-7341.  doi: 10.1016/j.jde.2019.11.064.  Google Scholar

[23]

W. Hahn, Stability of Motion, Springer, 1967.  Google Scholar

[24]

A. Porretta and E. Zuazua, Long time versus steady state optimal control, SIAM J. Control Optim., 51 (2013), 4242-4273.  doi: 10.1137/130907239.  Google Scholar

[25]

S. Primak, V. Kontorovich and V. Lyandres, Stochastic Methods and Their Applications to Communications, John Wiley & Sons, Inc., Hoboken, NJ, 2004.  Google Scholar

[26]

P. E. Protter, Stochastic Integration and Differential Equations, vol. 21 of Stochastic Modelling and Applied Probability, Springer-Verlag, Berlin, 2005. doi: 10.1007/978-3-662-10061-5.  Google Scholar

[27]

J. B. RawlingsD. BonnéJ. B. JørgensenA. N. Venkat and S. B. Jørgensen, Unreachable setpoints in model predictive control, IEEE Transactions on Automatic Control, 53 (2008), 2209-2215.  doi: 10.1109/TAC.2008.928125.  Google Scholar

[28]

J. B. Rawlings, D. Q. Mayne and M. M. Diehl, Model Predictive Control: Theory and Design, 2nd edition, Nob Hill Publishing, 2017. Google Scholar

[29]

H. Risken, The Fokker-Planck Equation, vol. 18 of Springer Series in Synergetics, 2nd edition, Springer-Verlag, Berlin, 1989. doi: 10.1007/978-3-642-61544-3.  Google Scholar

[30]

S. RoyM. Annunziato and A. Borzì, A Fokker-Planck feedback control-constrained approach for modelling crowd motion, J. Comput. Theor. Transp., 45 (2016), 442-458.  doi: 10.1080/23324309.2016.1189435.  Google Scholar

[31]

E. Trélat and E. Zuazua, The turnpike property in finite-dimensional nonlinear optimal control, J. Differential Equations, 258 (2015), 81-114.  doi: 10.1016/j.jde.2014.09.005.  Google Scholar

[32]

E. TrélatC. Zhang and E. Zuazua, Steady-state and periodic exponential turnpike property for optimal control problems in Hilbert spaces, SIAM J. Control Optim., 56 (2018), 1222-1252.  doi: 10.1137/16M1097638.  Google Scholar

[33]

F. Tröltzsch, Optimal Control of Partial Differential Equations, vol. 112 of Graduate Studies in Mathematics, American Mathematical Society, Providence, RI, 2010. doi: 10.1090/gsm/112.  Google Scholar

[34]

J. C. Willems, Dissipative dynamical systems. I. General theory, Arch. Rational Mech. Anal., 45 (1972), 321-351.  doi: 10.1007/BF00276493.  Google Scholar

Figure 1.  The state cost parts of the $ L^2 $, $ W^2 $, and 2F stage costs, i.e., (13) (in terms of $ \mu $ and $ \Sigma $ for $ d = 1 $), (15), and (16) for $ \gamma = 0 $, respectively. The desired state was set to $ (\bar{\mu},\bar{\Sigma}) = (0,1) $. The orange dot in the respective plots marks the minimum
Figure 2.  (Non-)Convexity of the reduced cost $ \hat{\ell}_{2F}(\Sigma,K) $ depending on $ \varsigma^2 $ (left) and on $ \gamma $ (right)
Figure 3.  Modified cost $ \tilde{\ell}_{L^2}(\Sigma,K) $ for Example 2. The optimal equilibrium $ (\Sigma^e,K^e) $ is illustrated by the orange circle. The white area represents negative values; the black diamond marks the minimum of the depicted area
Figure 4.  Modified cost $ \tilde{\ell}_{L^2}(\Sigma,K) $ for Example 3. The optimal equilibrium $ (\Sigma^e,K^e) $ is illustrated by the orange circle. The white area represents negative values; the black diamond marks the minimum of the depicted area
Figure 5.  Modified cost $ \tilde{\ell}_{W^2}(\Sigma,K) $ for Example 4 zoomed in (left) and zoomed out (right). The optimal equilibrium $ (\Sigma^e,K^e) $ is illustrated by the orange circle. The white area on the right plot is due to control constraints (26)
Figure 6.  Open loop optimal trajectories for various horizons $ N $ between $ 1 $ and $ 60 $ and MPC closed loop trajectories for two different initial conditions, indicating turnpike behavior in Example 2; state $ \Sigma $ (left) and control $ K $ (right)
Figure 7.  Open loop optimal trajectories for various horizons N between 1 and 60 and MPC closed loop trajectories for two different initial conditions, indicating turnpike behavior in Example 3; state Σ (left) and control K (right).
Figure 8.  New modified cost $ \tilde{\ell}_{W^2}^s(\Sigma,K) $ for Examples 2 (left) and 3 (right). The optimal equilibrium $ (\Sigma^e,K^e) $ is illustrated by the orange circle. The white area on the right plot is due to the control constraints (26)
[1]

Diana Keller. Optimal control of a linear stochastic Schrödinger equation. Conference Publications, 2013, 2013 (special) : 437-446. doi: 10.3934/proc.2013.2013.437

[2]

Paula A. González-Parra, Sunmi Lee, Leticia Velázquez, Carlos Castillo-Chavez. A note on the use of optimal control on a discrete time model of influenza dynamics. Mathematical Biosciences & Engineering, 2011, 8 (1) : 183-197. doi: 10.3934/mbe.2011.8.183

[3]

Shanjian Tang, Fu Zhang. Path-dependent optimal stochastic control and viscosity solution of associated Bellman equations. Discrete & Continuous Dynamical Systems - A, 2015, 35 (11) : 5521-5553. doi: 10.3934/dcds.2015.35.5521

[4]

J. Frédéric Bonnans, Justina Gianatti, Francisco J. Silva. On the convergence of the Sakawa-Shindo algorithm in stochastic control. Mathematical Control & Related Fields, 2016, 6 (3) : 391-406. doi: 10.3934/mcrf.2016008

[5]

Luke Finlay, Vladimir Gaitsgory, Ivan Lebedev. Linear programming solutions of periodic optimization problems: approximation of the optimal control. Journal of Industrial & Management Optimization, 2007, 3 (2) : 399-413. doi: 10.3934/jimo.2007.3.399

[6]

Guirong Jiang, Qishao Lu. The dynamics of a Prey-Predator model with impulsive state feedback control. Discrete & Continuous Dynamical Systems - B, 2006, 6 (6) : 1301-1320. doi: 10.3934/dcdsb.2006.6.1301

[7]

A. K. Misra, Anupama Sharma, Jia Li. A mathematical model for control of vector borne diseases through media campaigns. Discrete & Continuous Dynamical Systems - B, 2013, 18 (7) : 1909-1927. doi: 10.3934/dcdsb.2013.18.1909

[8]

Yves Dumont, Frederic Chiroleu. Vector control for the Chikungunya disease. Mathematical Biosciences & Engineering, 2010, 7 (2) : 313-345. doi: 10.3934/mbe.2010.7.313

[9]

Peter Benner, Jens Saak, M. Monir Uddin. Balancing based model reduction for structured index-2 unstable descriptor systems with application to flow control. Numerical Algebra, Control & Optimization, 2016, 6 (1) : 1-20. doi: 10.3934/naco.2016.6.1

[10]

Jan Prüss, Laurent Pujo-Menjouet, G.F. Webb, Rico Zacher. Analysis of a model for the dynamics of prions. Discrete & Continuous Dynamical Systems - B, 2006, 6 (1) : 225-235. doi: 10.3934/dcdsb.2006.6.225

[11]

Charles Fulton, David Pearson, Steven Pruess. Characterization of the spectral density function for a one-sided tridiagonal Jacobi matrix operator. Conference Publications, 2013, 2013 (special) : 247-257. doi: 10.3934/proc.2013.2013.247

[12]

Alberto Bressan, Ke Han, Franco Rampazzo. On the control of non holonomic systems by active constraints. Discrete & Continuous Dynamical Systems - A, 2013, 33 (8) : 3329-3353. doi: 10.3934/dcds.2013.33.3329

[13]

Martin Bohner, Sabrina Streipert. Optimal harvesting policy for the Beverton--Holt model. Mathematical Biosciences & Engineering, 2016, 13 (4) : 673-695. doi: 10.3934/mbe.2016014

[14]

Martial Agueh, Reinhard Illner, Ashlin Richardson. Analysis and simulations of a refined flocking and swarming model of Cucker-Smale type. Kinetic & Related Models, 2011, 4 (1) : 1-16. doi: 10.3934/krm.2011.4.1

[15]

Rui Hu, Yuan Yuan. Stability, bifurcation analysis in a neural network model with delay and diffusion. Conference Publications, 2009, 2009 (Special) : 367-376. doi: 10.3934/proc.2009.2009.367

[16]

Seung-Yeal Ha, Shi Jin. Local sensitivity analysis for the Cucker-Smale model with random inputs. Kinetic & Related Models, 2018, 11 (4) : 859-889. doi: 10.3934/krm.2018034

[17]

Nhu N. Nguyen, George Yin. Stochastic partial differential equation models for spatially dependent predator-prey equations. Discrete & Continuous Dynamical Systems - B, 2020, 25 (1) : 117-139. doi: 10.3934/dcdsb.2019175

[18]

Shihu Li, Wei Liu, Yingchao Xie. Large deviations for stochastic 3D Leray-$ \alpha $ model with fractional dissipation. Communications on Pure & Applied Analysis, 2019, 18 (5) : 2491-2509. doi: 10.3934/cpaa.2019113

[19]

Lakmi Niwanthi Wadippuli, Ivan Gudoshnikov, Oleg Makarenkov. Global asymptotic stability of nonconvex sweeping processes. Discrete & Continuous Dynamical Systems - B, 2020, 25 (3) : 1129-1139. doi: 10.3934/dcdsb.2019212

[20]

Sara Munday. On the derivative of the $\alpha$-Farey-Minkowski function. Discrete & Continuous Dynamical Systems - A, 2014, 34 (2) : 709-732. doi: 10.3934/dcds.2014.34.709

2019 Impact Factor: 0.857

Metrics

  • PDF downloads (26)
  • HTML views (95)
  • Cited by (0)

Other articles
by authors

[Back to Top]