# American Institute of Mathematical Sciences

March  2022, 14(1): 131-150. doi: 10.3934/jgm.2021032

## Modeling student engagement using optimal control theory

 Mathematics Department, University of California Santa Cruz, Santa Cruz, CA 95064, USA

* Corresponding author: Debra Lewis

Dedicated to Professor Anthony Bloch on the occasion of his 65th birthday.

Received  June 2021 Revised  November 2021 Published  March 2022 Early access  January 2022

Student engagement in learning a prescribed body of knowledge can be modeled using optimal control theory, with a scalar state variable representing mastery, or self-perceived mastery, of the material and control representing the instantaneous cognitive effort devoted to the learning task. The relevant costs include emotional and external penalties for incomplete mastery, reduced availability of cognitive resources for other activities, and psychological stresses related to engagement with the learning task. Application of Pontryagin's maximum principle to some simple models of engagement yields solutions of the synthesis problem mimicking familiar behaviors including avoidance, procrastination, and increasing commitment in response to increasing mastery.

Citation: Debra Lewis. Modeling student engagement using optimal control theory. Journal of Geometric Mechanics, 2022, 14 (1) : 131-150. doi: 10.3934/jgm.2021032
##### References:
 [1] M. C. Anderson, R. A. Bjork and E. L. Bjork, Remembering can cause forgetting: Retrieval dynamics in long-term memory, Journal of Experimental Psychology. Learning, Memory, and Cognition, 20 (1994), 1063-1087.  doi: 10.1037/0278-7393.20.5.1063. [2] L. Averell and A. Heathcote, The form of the forgetting curve and the fate of memories, Journal of Mathematical Psychology, 55 (2011), 25-35.  doi: 10.1016/j.jmp.2010.08.009. [3] J. Baillieul and J. Willems, Mathematical Control Theory, Springer, 1999. doi: 10.1007/978-1-4612-1416-8. [4] A. Bandura, Self-Efficacy in Changing Societies, Cambridge University Press, 1995. [5] A. M. Bloch, Nonholonomic Mechanics and Control, Springer, 2003. doi: 10.1007/b97376. [6] H. Chang and S. L. Beilock, The math anxiety-math performance link and its relation to individual and environmental factors: a review of current behavioral and psychophysiological research, Current Opinion in Behavioral Sciences, 10 (2016), 33-38.  doi: 10.1016/j.cobeha.2016.04.011. [7] K. Choe, J. Jenifer, C. Rozek, M. Berman and S. Beilock, Calculated avoidance: Math anxiety predicts math avoidance in effort-based decision-making, Science Advances, 5 (2019).  doi: 10.1126/sciadv.aay1062. [8] R. Dorfman, An economic interpretation of optimal control theory, The American Economic Review, 59 (1969), 817-831. [9] H. Ebbinghaus, Über das Gedächtnis, Dunker, 1885. [10] E. O. Finkenbinder, The curve of forgetting, The American Journal of Psychology, 24 (1913), 8-32.  doi: 10.2307/1413271. [11] T. L. Griffiths, F. Lieder and N. D. Goodman, Rational use of cognitive resources: Levels of analysis between the computational and the algorithmic, Topics in Cognitive Science, 11 (2015), 217-229.  doi: 10.1111/tops.12142. [12] K. Hall, E. Fawcett, K. Hourihan and J. Fawcett, Emotional memories are (usually) harder to forget: A meta-analysis of the item-method directed forgetting literature, Psychonomic Bulletin & Review, 28 (2021), 1313-1326.  doi: 10.3758/s13423-021-01914-z. [13] D. Kirk, Optimal Control Theory: An Introduction, Dover Publications, Inc., 2004. [14] S. Koessler, H. Engler, C. Riether and J. Kissler, No retrieval-induced forgetting under stress, Psychological Science, 20 (2009), 1356-1363.  doi: 10.1111/j.1467-9280.2009.02450.x. [15] D. Lewis, A soothing invisible hand: Moderation potentials in optimal control, in Geometry, Mechanics, and Dynamics: The Legacy of Jerry Marsden, Springer, 2015,257–284. doi: 10.1007/978-1-4939-2441-7_12. [16] G. R. Loftus, Evaluating forgetting curves, Journal of Experimental Psychology, 11 (1985), 397-406.  doi: 10.1037/0278-7393.11.2.397. [17] I. Lyons and S. Beilock, When math hurts: Math anxiety predicts pain network activation in anticipation of doing math, PLOS ONE, 7. doi: 10.1371/journal.pone.0048076. [18] E. Maloney, M. Schaeffer and S. Beilock, Mathematics anxiety and stereotype threat: Shared mechanisms, negative consequences, and promising interventions, Research in Mathematics Education, 15 (2013), 115-128.  doi: 10.1080/14794802.2013.797744. [19] A. Mattarella-Micke, J. Mateo, M. Kozak, K. Foster and S. Beilock, Choke or thrive? The relation between salivary cortisol and math performance depends on individual differences in working memory and math anxiety, Emotion, 11 (2011), 1000-1005.  doi: 10.1037/a0023224. [20] H. Melville, Bartleby, the scrivener: A story of Wall Street, Putnam's Magazine, November and December (1853), 446–457 and 609–615. doi: 10.1093/oseo/instance.00209193. [21] R. Montgomery, Optimal control of deformable bodies and its relation to gauge theory, Math. Sci. Res. Inst. Publ., 22 (1991), 403-438.  doi: 10.1007/978-1-4613-9725-0_15. [22] J. Murre and J. Dros, Replication and analysis of Ebbinghaus' forgetting curve, PLOS ONE, 10 (2015).  doi: 10.1371/journal.pone.0120644. [23] L. Pontryagin, V. Boltyanskii, R. Gamkrelidze and E. Mishchenko, The Mathematical Theory of Optimal Processes, Interscience Publishers, 1962. [24] R. Rydell, A. McConnell and S. Beilock, Multiple social identities and stereotype threat: Imbalance, accessibility, and working memory, Journal of Personality and Social Psychology, 96 (2009), 949-966.  doi: 10.1037/a0014846. [25] A. Shenhav, S. Musslick, F. Lieder, W. Kool, T. Griffiths, J. Cohen and M. Botvinick, Toward a rational and mechanistic account of mental effort, Annual Review of Neuroscience, 40 (2017), 99-124.  doi: 10.1146/annurev-neuro-072116-031526. [26] F. Sirois and T. Pychyl, Procrastination and the priority of short-term mood regulation: Consequences for future self, Social and Personality Psychology Compass, 7 (2013), 115-127.  doi: 10.1111/spc3.12011. [27] N. Slamecka, On comparing rates of forgetting: Comment on Loftus (1985), Journal of Experimental Psychology. Learning, Memory, and Cognition, 11 (1985), 812-816.  doi: 10.1037/0278-7393.11.1-4.812. [28] E. Sontag, Integrability of certain distributions associated with actions on manifolds and applications to control problems, in Nonlinear Controlability and Optimal Control, Marcel Dekker, Inc., 133 (1990), 81–131.

show all references

##### References:
 [1] M. C. Anderson, R. A. Bjork and E. L. Bjork, Remembering can cause forgetting: Retrieval dynamics in long-term memory, Journal of Experimental Psychology. Learning, Memory, and Cognition, 20 (1994), 1063-1087.  doi: 10.1037/0278-7393.20.5.1063. [2] L. Averell and A. Heathcote, The form of the forgetting curve and the fate of memories, Journal of Mathematical Psychology, 55 (2011), 25-35.  doi: 10.1016/j.jmp.2010.08.009. [3] J. Baillieul and J. Willems, Mathematical Control Theory, Springer, 1999. doi: 10.1007/978-1-4612-1416-8. [4] A. Bandura, Self-Efficacy in Changing Societies, Cambridge University Press, 1995. [5] A. M. Bloch, Nonholonomic Mechanics and Control, Springer, 2003. doi: 10.1007/b97376. [6] H. Chang and S. L. Beilock, The math anxiety-math performance link and its relation to individual and environmental factors: a review of current behavioral and psychophysiological research, Current Opinion in Behavioral Sciences, 10 (2016), 33-38.  doi: 10.1016/j.cobeha.2016.04.011. [7] K. Choe, J. Jenifer, C. Rozek, M. Berman and S. Beilock, Calculated avoidance: Math anxiety predicts math avoidance in effort-based decision-making, Science Advances, 5 (2019).  doi: 10.1126/sciadv.aay1062. [8] R. Dorfman, An economic interpretation of optimal control theory, The American Economic Review, 59 (1969), 817-831. [9] H. Ebbinghaus, Über das Gedächtnis, Dunker, 1885. [10] E. O. Finkenbinder, The curve of forgetting, The American Journal of Psychology, 24 (1913), 8-32.  doi: 10.2307/1413271. [11] T. L. Griffiths, F. Lieder and N. D. Goodman, Rational use of cognitive resources: Levels of analysis between the computational and the algorithmic, Topics in Cognitive Science, 11 (2015), 217-229.  doi: 10.1111/tops.12142. [12] K. Hall, E. Fawcett, K. Hourihan and J. Fawcett, Emotional memories are (usually) harder to forget: A meta-analysis of the item-method directed forgetting literature, Psychonomic Bulletin & Review, 28 (2021), 1313-1326.  doi: 10.3758/s13423-021-01914-z. [13] D. Kirk, Optimal Control Theory: An Introduction, Dover Publications, Inc., 2004. [14] S. Koessler, H. Engler, C. Riether and J. Kissler, No retrieval-induced forgetting under stress, Psychological Science, 20 (2009), 1356-1363.  doi: 10.1111/j.1467-9280.2009.02450.x. [15] D. Lewis, A soothing invisible hand: Moderation potentials in optimal control, in Geometry, Mechanics, and Dynamics: The Legacy of Jerry Marsden, Springer, 2015,257–284. doi: 10.1007/978-1-4939-2441-7_12. [16] G. R. Loftus, Evaluating forgetting curves, Journal of Experimental Psychology, 11 (1985), 397-406.  doi: 10.1037/0278-7393.11.2.397. [17] I. Lyons and S. Beilock, When math hurts: Math anxiety predicts pain network activation in anticipation of doing math, PLOS ONE, 7. doi: 10.1371/journal.pone.0048076. [18] E. Maloney, M. Schaeffer and S. Beilock, Mathematics anxiety and stereotype threat: Shared mechanisms, negative consequences, and promising interventions, Research in Mathematics Education, 15 (2013), 115-128.  doi: 10.1080/14794802.2013.797744. [19] A. Mattarella-Micke, J. Mateo, M. Kozak, K. Foster and S. Beilock, Choke or thrive? The relation between salivary cortisol and math performance depends on individual differences in working memory and math anxiety, Emotion, 11 (2011), 1000-1005.  doi: 10.1037/a0023224. [20] H. Melville, Bartleby, the scrivener: A story of Wall Street, Putnam's Magazine, November and December (1853), 446–457 and 609–615. doi: 10.1093/oseo/instance.00209193. [21] R. Montgomery, Optimal control of deformable bodies and its relation to gauge theory, Math. Sci. Res. Inst. Publ., 22 (1991), 403-438.  doi: 10.1007/978-1-4613-9725-0_15. [22] J. Murre and J. Dros, Replication and analysis of Ebbinghaus' forgetting curve, PLOS ONE, 10 (2015).  doi: 10.1371/journal.pone.0120644. [23] L. Pontryagin, V. Boltyanskii, R. Gamkrelidze and E. Mishchenko, The Mathematical Theory of Optimal Processes, Interscience Publishers, 1962. [24] R. Rydell, A. McConnell and S. Beilock, Multiple social identities and stereotype threat: Imbalance, accessibility, and working memory, Journal of Personality and Social Psychology, 96 (2009), 949-966.  doi: 10.1037/a0014846. [25] A. Shenhav, S. Musslick, F. Lieder, W. Kool, T. Griffiths, J. Cohen and M. Botvinick, Toward a rational and mechanistic account of mental effort, Annual Review of Neuroscience, 40 (2017), 99-124.  doi: 10.1146/annurev-neuro-072116-031526. [26] F. Sirois and T. Pychyl, Procrastination and the priority of short-term mood regulation: Consequences for future self, Social and Personality Psychology Compass, 7 (2013), 115-127.  doi: 10.1111/spc3.12011. [27] N. Slamecka, On comparing rates of forgetting: Comment on Loftus (1985), Journal of Experimental Psychology. Learning, Memory, and Cognition, 11 (1985), 812-816.  doi: 10.1037/0278-7393.11.1-4.812. [28] E. Sontag, Integrability of certain distributions associated with actions on manifolds and applications to control problems, in Nonlinear Controlability and Optimal Control, Marcel Dekker, Inc., 133 (1990), 81–131.
Left: graphs of $u_c(\mu, \cdot )$ for sample values of $\mu$; the lowest curve is the graph of $u_c(1, \cdot )$. Right: contour plot of $u_{\rm opt}$, with contour values $\frac j {10}, j = 0, \ldots, 10$. Black: u = 0; white: u = 1
Left: $K_{\frac \pi {10}}(r)$ for $r$ near $\tan \frac \pi {10}$, right: $\upsilon_\circ(r)$ associated to $\xi_\circ(u) = \sqrt{1 - u^2}$
Graphs of $X_{h_0}^\pm$ for $\mathcal{S} = [0, 1]$, $\gamma(m) = (1 - m)^2$, $\mu(m) = {1 \over 2} \left ( 1 - \frac m 2 \right )^2$, constant functions $\psi_{\rm pe} = {1 \over 2}$ and $\phi = \frac \pi {12}$, and representative values of $h_0$ for which $\breve C_{h_0}$ has a strict global minimum ${c_{h_0}}$. Solid curves: $X_{h_0}^+$; dashed curves: $X_{h_0}^-$
Representative vertically aligned graphs of $K_\phi(r)$ and $\breve C_{h_0}(m)$ for constant $\phi$ and three values of $h_0$. Solid: projections of the trajectories with initial data $(m_0, r_0)$; dotted: points on the graphs outside those projections. Upper row: $r_0 < r_\phi$ and $m_0 > m_c$; initially, $m$ decreases and $r$ increases. Lower row: $r_0 > r_\phi$ and $m_0 < m_c$; initially, $m$ increases and $r$ decreases. Left: $m'$ changes sign as $r$ passes through $r_\phi$; center: $m$ and $r$ asymptotically approach the equilibrium $(m_c, r_\phi)$; right: $r'$ changes sign as $m$ passes through $m_c$
 [1] Hans Josef Pesch. Carathéodory's royal road of the calculus of variations: Missed exits to the maximum principle of optimal control theory. Numerical Algebra, Control and Optimization, 2013, 3 (1) : 161-173. doi: 10.3934/naco.2013.3.161 [2] Stefan Doboszczak, Manil T. Mohan, Sivaguru S. Sritharan. Pontryagin maximum principle for the optimal control of linearized compressible navier-stokes equations with state constraints. Evolution Equations and Control Theory, 2022, 11 (2) : 347-371. doi: 10.3934/eect.2020110 [3] Zaidong Zhan, Shuping Chen, Wei Wei. A unified theory of maximum principle for continuous and discrete time optimal control problems. Mathematical Control and Related Fields, 2012, 2 (2) : 195-215. doi: 10.3934/mcrf.2012.2.195 [4] Huaiqiang Yu, Bin Liu. Pontryagin's principle for local solutions of optimal control governed by the 2D Navier-Stokes equations with mixed control-state constraints. Mathematical Control and Related Fields, 2012, 2 (1) : 61-80. doi: 10.3934/mcrf.2012.2.61 [5] Torsten Lindström. Discrete models and Fisher's maximum principle in ecology. Conference Publications, 2003, 2003 (Special) : 571-579. doi: 10.3934/proc.2003.2003.571 [6] Guy Barles, Ariela Briani, Emmanuel Trélat. Value function for regional control problems via dynamic programming and Pontryagin maximum principle. Mathematical Control and Related Fields, 2018, 8 (3&4) : 509-533. doi: 10.3934/mcrf.2018021 [7] Xiao-Li Ding, Iván Area, Juan J. Nieto. Controlled singular evolution equations and Pontryagin type maximum principle with applications. Evolution Equations and Control Theory, 2021  doi: 10.3934/eect.2021059 [8] Martin Benning, Elena Celledoni, Matthias J. Ehrhardt, Brynjulf Owren, Carola-Bibiane Schönlieb. Deep learning as optimal control problems: Models and numerical methods. Journal of Computational Dynamics, 2019, 6 (2) : 171-198. doi: 10.3934/jcd.2019009 [9] Md. Haider Ali Biswas, Maria do Rosário de Pinho. A nonsmooth maximum principle for optimal control problems with state and mixed constraints - convex case. Conference Publications, 2011, 2011 (Special) : 174-183. doi: 10.3934/proc.2011.2011.174 [10] Zhen Wu, Feng Zhang. Maximum principle for discrete-time stochastic optimal control problem and stochastic game. Mathematical Control and Related Fields, 2022, 12 (2) : 475-493. doi: 10.3934/mcrf.2021031 [11] Omid S. Fard, Javad Soolaki, Delfim F. M. Torres. A necessary condition of Pontryagin type for fuzzy fractional optimal control problems. Discrete and Continuous Dynamical Systems - S, 2018, 11 (1) : 59-76. doi: 10.3934/dcdss.2018004 [12] Hartmut Schwetlick, Daniel C. Sutton, Johannes Zimmer. Effective Hamiltonian dynamics via the Maupertuis principle. Discrete and Continuous Dynamical Systems - S, 2020, 13 (4) : 1395-1410. doi: 10.3934/dcdss.2020078 [13] G. Calafiore, M.C. Campi. A learning theory approach to the construction of predictor models. Conference Publications, 2003, 2003 (Special) : 156-166. doi: 10.3934/proc.2003.2003.156 [14] Yan Wang, Yanxiang Zhao, Lei Wang, Aimin Song, Yanping Ma. Stochastic maximum principle for partial information optimal investment and dividend problem of an insurer. Journal of Industrial and Management Optimization, 2018, 14 (2) : 653-671. doi: 10.3934/jimo.2017067 [15] Shanjian Tang. A second-order maximum principle for singular optimal stochastic controls. Discrete and Continuous Dynamical Systems - B, 2010, 14 (4) : 1581-1599. doi: 10.3934/dcdsb.2010.14.1581 [16] H. O. Fattorini. The maximum principle for linear infinite dimensional control systems with state constraints. Discrete and Continuous Dynamical Systems, 1995, 1 (1) : 77-101. doi: 10.3934/dcds.1995.1.77 [17] Shaolin Ji, Xiaole Xue. A stochastic maximum principle for linear quadratic problem with nonconvex control domain. Mathematical Control and Related Fields, 2019, 9 (3) : 495-507. doi: 10.3934/mcrf.2019022 [18] H. O. Fattorini. The maximum principle in infinite dimension. Discrete and Continuous Dynamical Systems, 2000, 6 (3) : 557-574. doi: 10.3934/dcds.2000.6.557 [19] Antonio Fernández, Pedro L. García. Regular discretizations in optimal control theory. Journal of Geometric Mechanics, 2013, 5 (4) : 415-432. doi: 10.3934/jgm.2013.5.415 [20] K. Renee Fister, Jennifer Hughes Donnelly. Immunotherapy: An Optimal Control Theory Approach. Mathematical Biosciences & Engineering, 2005, 2 (3) : 499-510. doi: 10.3934/mbe.2005.2.499

2020 Impact Factor: 0.857