
-
Previous Article
Augmented Gaussian random field: Theory and computation
- DCDS-S Home
- This Issue
-
Next Article
A stochastic collocation method based on sparse grids for a stochastic Stokes-Darcy model
An out-of-distribution-aware autoencoder model for reduced chemical kinetics
1. | Computational Sciences and Engineering Division, Oak Ridge National Laboratory, Oak Ridge, TN 37830, USA |
2. | Computer Science and Mathematics Division, Oak Ridge National Laboratory, Oak Ridge, TN 37830, USA |
While detailed chemical kinetic models have been successful in representing rates of chemical reactions in continuum scale computational fluid dynamics (CFD) simulations, applying the models in simulations for engineering device conditions is computationally prohibitive. To reduce the cost, data-driven methods, e.g., autoencoders, have been used to construct reduced chemical kinetic models for CFD simulations. Despite their success, data-driven methods rely heavily on training data sets and can be unreliable when used in out-of-distribution (OOD) regions (i.e., when extrapolating outside of the training set). In this paper, we present an enhanced autoencoder model for combustion chemical kinetics with uncertainty quantification to enable the detection of model usage in OOD regions, and thereby creating an OOD-aware autoencoder model that contributes to more robust CFD simulations of reacting flows. We first demonstrate the effectiveness of the method in OOD detection in two well-known datasets, MNIST and Fashion-MNIST, in comparison with the deep ensemble method, and then present the OOD-aware autoencoder for reduced chemistry model in syngas combustion.
References:
[1] |
A. Amini, W. Schwarting, A. Soleimany and D. Rus,
Deep evidential regression, Advances in Neural Information Processing Systems, 33 (2020), 14927-14937.
|
[2] |
R. B. Bird, W. E. Stewart and E. N. Lightfoot, Transport Phenomena, John Wiley and Sons, New York, 1960. |
[3] |
J. H. Chen, A. Choudhary, B. de Supinski, M. DeVries, E. R. Hawkes, S. Klasky, W. K. Liao, K. L. Ma, J. Mellor-Crummey, N. Podhorszki, R. Sankaran, S. Shende and C. S. Yoo,
Terascale direct numerical simulations of turbulent combustion using S3D, Computational Science & Discovery, 2 (2009), 015001.
doi: 10.1088/1749-4699/2/1/015001. |
[4] |
A. Coussement, O. Gicquel and A. Parente,
MG-local-PCA method for reduced order combustion modeling, Proceedings of the Combustion Institute, 34 (2013), 1117-1123.
doi: 10.1016/j.proci.2012.05.073. |
[5] |
G. Esposito and H. Chelliah,
Skeletal reaction models based on principal component analysis: Application to ethylene-air ignition, propagation, and extinction phenomena, Combustion and Flame, 158 (2011), 477-489.
doi: 10.1016/j.combustflame.2010.09.010. |
[6] |
Y. Gal and Z. Ghahramani,
Dropout as a Bayesian approximation: Representing model uncertainty in deep learning, Proceedings of The 33rd International Conference on Machine Learning, 48 (2016), 1050-1059.
|
[7] |
E. R. Hawkes, R. Sankaran, J. C. Sutherland and J. H. Chen,
Scalar mixing in direct numerical simulations of temporally evolving plane jet flames with skeletal {CO/H$_2$} kinetics, Proceedings of the Combustion Institute, 31 (2007), 1633-1640.
|
[8] |
M. D. Hoffman, D. M. Blei, C. Wang and J. Paisley, Stochastic variational inference, J. Mach. Learn. Res., 14 (2013), 1303–1347, URL http://jmlr.org/papers/v14/hoffman13a.html. |
[9] |
B. J. Isaac, A. Coussement, O. Gicquel, P. J. Smith and A. Parente,
Reduced-order PCA models for chemical reacting flows, Combustion and Flame, 161 (2014), 2785-2800.
doi: 10.1016/j.combustflame.2014.05.011. |
[10] |
B. J. Isaac, J. N. Thornock, J. Sutherland, P. J. Smith and A. Parente,
Advanced regression methods for combustion modelling using principal components, Combustion and Flame, 162 (2015), 2592-2601.
doi: 10.1016/j.combustflame.2015.03.008. |
[11] |
B. Lakshminarayanan, A. Pritzel and C. Blundell,
Simple and scalable predictive uncertainty estimation using deep ensembles, Proceedings of the 31st International Conference on Neural Information Processing Systems, 17 (2017), 6405-6416.
|
[12] |
Y. Li, C.-W. Zhou, K. P. Somers, K. Zhang and H. J. Curran,
The oxidation of 2-butene: A high pressure ignition delay, kinetic modeling study and reactivity comparison with isobutene and 1-butene, Proceedings of the Combustion Institute, 36 (2017), 403-411.
doi: 10.1016/j.proci.2016.05.052. |
[13] |
T. Lu and C. K. Law,
Toward accommodating realistic fuel chemistry in large-scale computations, Progress in Energy and Combustion Science, 35 (2009), 192-215.
doi: 10.1016/j.pecs.2008.10.002. |
[14] |
D. J. C. MacKay,
A practical Bayesian framework for backpropagation networks, Neural Comput., 4 (1992), 448-472.
doi: 10.1162/neco.1992.4.3.448. |
[15] |
M. R. Malik, B. J. Isaac, A. Coussement, P. J. Smith and A. Parente,
Principal component analysis coupled with nonlinear regression for chemistry reduction, Combustion and Flame, 187 (2018), 30-41.
doi: 10.1016/j.combustflame.2017.08.012. |
[16] |
H. Mirgolbabaei and T. Echekki,
A novel principal component analysis-based acceleration scheme for LES–ODT: An a priori study, Combustion and Flame, 160 (2013), 898-908.
doi: 10.2514/6.2013-168. |
[17] |
H. Mirgolbabaei and T. Echekki,
Nonlinear reduction of combustion composition space with kernel principal component analysis, Combustion and Flame, 161 (2014), 118-126.
doi: 10.1016/j.combustflame.2013.08.016. |
[18] |
H. Mirgolbabaei and T. Echekki,
The reconstruction of thermo-chemical scalars in combustion from a reduced set of their principal components, Combustion and Flame, 162 (2015), 1650-1652.
doi: 10.1016/j.combustflame.2014.11.027. |
[19] |
H. Mirgolbabaei, T. Echekki and N. Smaoui,
A nonlinear principal component analysis approach for turbulent combustion composition space, International Journal of Hydrogen Energy, 39 (2014), 4622-4633.
doi: 10.1016/j.ijhydene.2013.12.195. |
[20] |
D. A. Nix and A. S. Weigend, Learning local error bars for nonlinear regression, In Advances in neural information processing systems, (1995), 489–496. |
[21] |
A. Parente, J. C. Sutherland, L. Tognotti and P. J. Smith,
Identification of low-dimensional manifolds in turbulent flames, Proceedings of the Combustion Institute, 32 (2009), 1579-1586.
doi: 10.1016/j.proci.2008.06.177. |
[22] |
T. Pearce, A. Brintrup, M. Zaki and A. Neely,
High-quality prediction intervals for deep learning: A distribution-free, ensembled approach, Proceedings of the 35th International Conference on Machine Learning, 80 (2018), 4075-4084.
|
[23] |
T. Pearce, F. Leibfried and A. Brintrup, Uncertainty in neural networks: Approximately Bayesian ensembling, In Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, Proceedings of Machine Learning Research, 108 (2020), 234–244. |
[24] |
T. S. Salem, H. Langseth and H. Ramampiaro, Prediction intervals: Split normal mixture from quality-driven deep ensembles, In Proceedings of the 36th Conference on Uncertainty in Artificial Intelligence (UAI), Proceedings of Machine Learning Research, 124 (2020), 1179–1187. |
[25] |
R. Shan and T. Lu,
Ignition and extinction in perfectly stirred reactors with detailed chemistry, Combustion and Flame, 159 (2012), 2069-2076.
doi: 10.1016/j.combustflame.2012.01.023. |
[26] |
E. Simhayev, G. Katz and L. Rokach, PIVEN: A deep neural network for prediction intervals with specific value prediction, 2021. |
[27] |
J. C. Sutherland and A. Parente,
Combustion modeling using principal component analysis, Proceedings of the Combustion Institute, 32 (2009), 1563-1570.
|
[28] |
S. Vajda, P. Valko and T. Turányi,
Principal component analysis of kinetic models, International Journal of Chemical Kinetics, 17 (1985), 55-81.
doi: 10.1002/kin.550170107. |
[29] |
P. Zhang, S. Liu, D. Lu, G. Zhang and R. Sankaran, A prediction interval method for uncertainty quantification of regression models, In ICLR 2021 SimDL Workshop, Virtual, 2021. |
[30] |
P. Zhang, R. Sankaran and E. R. Hawkes, A priori examination of reduced chemistry models derived from canonical stirred reactors using three-dimensional direct numerical simulation datasets, In AIAA Scitech, (2021), 1784
doi: 10.2514/6.2021-1784. |
[31] |
P. Zhang, R. Sankaran, M. Stoyanov, D. Lebrun-Grandie and C. E. Finney, Reduced models for chemical kinetics derived from parallel ensemble simulations of stirred reactors, In AIAA Scitech Forum, (2020), 0177.
doi: 10.2514/6.2020-0177. |
show all references
References:
[1] |
A. Amini, W. Schwarting, A. Soleimany and D. Rus,
Deep evidential regression, Advances in Neural Information Processing Systems, 33 (2020), 14927-14937.
|
[2] |
R. B. Bird, W. E. Stewart and E. N. Lightfoot, Transport Phenomena, John Wiley and Sons, New York, 1960. |
[3] |
J. H. Chen, A. Choudhary, B. de Supinski, M. DeVries, E. R. Hawkes, S. Klasky, W. K. Liao, K. L. Ma, J. Mellor-Crummey, N. Podhorszki, R. Sankaran, S. Shende and C. S. Yoo,
Terascale direct numerical simulations of turbulent combustion using S3D, Computational Science & Discovery, 2 (2009), 015001.
doi: 10.1088/1749-4699/2/1/015001. |
[4] |
A. Coussement, O. Gicquel and A. Parente,
MG-local-PCA method for reduced order combustion modeling, Proceedings of the Combustion Institute, 34 (2013), 1117-1123.
doi: 10.1016/j.proci.2012.05.073. |
[5] |
G. Esposito and H. Chelliah,
Skeletal reaction models based on principal component analysis: Application to ethylene-air ignition, propagation, and extinction phenomena, Combustion and Flame, 158 (2011), 477-489.
doi: 10.1016/j.combustflame.2010.09.010. |
[6] |
Y. Gal and Z. Ghahramani,
Dropout as a Bayesian approximation: Representing model uncertainty in deep learning, Proceedings of The 33rd International Conference on Machine Learning, 48 (2016), 1050-1059.
|
[7] |
E. R. Hawkes, R. Sankaran, J. C. Sutherland and J. H. Chen,
Scalar mixing in direct numerical simulations of temporally evolving plane jet flames with skeletal {CO/H$_2$} kinetics, Proceedings of the Combustion Institute, 31 (2007), 1633-1640.
|
[8] |
M. D. Hoffman, D. M. Blei, C. Wang and J. Paisley, Stochastic variational inference, J. Mach. Learn. Res., 14 (2013), 1303–1347, URL http://jmlr.org/papers/v14/hoffman13a.html. |
[9] |
B. J. Isaac, A. Coussement, O. Gicquel, P. J. Smith and A. Parente,
Reduced-order PCA models for chemical reacting flows, Combustion and Flame, 161 (2014), 2785-2800.
doi: 10.1016/j.combustflame.2014.05.011. |
[10] |
B. J. Isaac, J. N. Thornock, J. Sutherland, P. J. Smith and A. Parente,
Advanced regression methods for combustion modelling using principal components, Combustion and Flame, 162 (2015), 2592-2601.
doi: 10.1016/j.combustflame.2015.03.008. |
[11] |
B. Lakshminarayanan, A. Pritzel and C. Blundell,
Simple and scalable predictive uncertainty estimation using deep ensembles, Proceedings of the 31st International Conference on Neural Information Processing Systems, 17 (2017), 6405-6416.
|
[12] |
Y. Li, C.-W. Zhou, K. P. Somers, K. Zhang and H. J. Curran,
The oxidation of 2-butene: A high pressure ignition delay, kinetic modeling study and reactivity comparison with isobutene and 1-butene, Proceedings of the Combustion Institute, 36 (2017), 403-411.
doi: 10.1016/j.proci.2016.05.052. |
[13] |
T. Lu and C. K. Law,
Toward accommodating realistic fuel chemistry in large-scale computations, Progress in Energy and Combustion Science, 35 (2009), 192-215.
doi: 10.1016/j.pecs.2008.10.002. |
[14] |
D. J. C. MacKay,
A practical Bayesian framework for backpropagation networks, Neural Comput., 4 (1992), 448-472.
doi: 10.1162/neco.1992.4.3.448. |
[15] |
M. R. Malik, B. J. Isaac, A. Coussement, P. J. Smith and A. Parente,
Principal component analysis coupled with nonlinear regression for chemistry reduction, Combustion and Flame, 187 (2018), 30-41.
doi: 10.1016/j.combustflame.2017.08.012. |
[16] |
H. Mirgolbabaei and T. Echekki,
A novel principal component analysis-based acceleration scheme for LES–ODT: An a priori study, Combustion and Flame, 160 (2013), 898-908.
doi: 10.2514/6.2013-168. |
[17] |
H. Mirgolbabaei and T. Echekki,
Nonlinear reduction of combustion composition space with kernel principal component analysis, Combustion and Flame, 161 (2014), 118-126.
doi: 10.1016/j.combustflame.2013.08.016. |
[18] |
H. Mirgolbabaei and T. Echekki,
The reconstruction of thermo-chemical scalars in combustion from a reduced set of their principal components, Combustion and Flame, 162 (2015), 1650-1652.
doi: 10.1016/j.combustflame.2014.11.027. |
[19] |
H. Mirgolbabaei, T. Echekki and N. Smaoui,
A nonlinear principal component analysis approach for turbulent combustion composition space, International Journal of Hydrogen Energy, 39 (2014), 4622-4633.
doi: 10.1016/j.ijhydene.2013.12.195. |
[20] |
D. A. Nix and A. S. Weigend, Learning local error bars for nonlinear regression, In Advances in neural information processing systems, (1995), 489–496. |
[21] |
A. Parente, J. C. Sutherland, L. Tognotti and P. J. Smith,
Identification of low-dimensional manifolds in turbulent flames, Proceedings of the Combustion Institute, 32 (2009), 1579-1586.
doi: 10.1016/j.proci.2008.06.177. |
[22] |
T. Pearce, A. Brintrup, M. Zaki and A. Neely,
High-quality prediction intervals for deep learning: A distribution-free, ensembled approach, Proceedings of the 35th International Conference on Machine Learning, 80 (2018), 4075-4084.
|
[23] |
T. Pearce, F. Leibfried and A. Brintrup, Uncertainty in neural networks: Approximately Bayesian ensembling, In Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, Proceedings of Machine Learning Research, 108 (2020), 234–244. |
[24] |
T. S. Salem, H. Langseth and H. Ramampiaro, Prediction intervals: Split normal mixture from quality-driven deep ensembles, In Proceedings of the 36th Conference on Uncertainty in Artificial Intelligence (UAI), Proceedings of Machine Learning Research, 124 (2020), 1179–1187. |
[25] |
R. Shan and T. Lu,
Ignition and extinction in perfectly stirred reactors with detailed chemistry, Combustion and Flame, 159 (2012), 2069-2076.
doi: 10.1016/j.combustflame.2012.01.023. |
[26] |
E. Simhayev, G. Katz and L. Rokach, PIVEN: A deep neural network for prediction intervals with specific value prediction, 2021. |
[27] |
J. C. Sutherland and A. Parente,
Combustion modeling using principal component analysis, Proceedings of the Combustion Institute, 32 (2009), 1563-1570.
|
[28] |
S. Vajda, P. Valko and T. Turányi,
Principal component analysis of kinetic models, International Journal of Chemical Kinetics, 17 (1985), 55-81.
doi: 10.1002/kin.550170107. |
[29] |
P. Zhang, S. Liu, D. Lu, G. Zhang and R. Sankaran, A prediction interval method for uncertainty quantification of regression models, In ICLR 2021 SimDL Workshop, Virtual, 2021. |
[30] |
P. Zhang, R. Sankaran and E. R. Hawkes, A priori examination of reduced chemistry models derived from canonical stirred reactors using three-dimensional direct numerical simulation datasets, In AIAA Scitech, (2021), 1784
doi: 10.2514/6.2021-1784. |
[31] |
P. Zhang, R. Sankaran, M. Stoyanov, D. Lebrun-Grandie and C. E. Finney, Reduced models for chemical kinetics derived from parallel ensemble simulations of stirred reactors, In AIAA Scitech Forum, (2020), 0177.
doi: 10.2514/6.2020-0177. |








[1] |
Andrew J. Majda, Michal Branicki. Lessons in uncertainty quantification for turbulent dynamical systems. Discrete and Continuous Dynamical Systems, 2012, 32 (9) : 3133-3221. doi: 10.3934/dcds.2012.32.3133 |
[2] |
Jing Li, Panos Stinis. Mori-Zwanzig reduced models for uncertainty quantification. Journal of Computational Dynamics, 2019, 6 (1) : 39-68. doi: 10.3934/jcd.2019002 |
[3] |
H. T. Banks, Robert Baraldi, Karissa Cross, Kevin Flores, Christina McChesney, Laura Poag, Emma Thorpe. Uncertainty quantification in modeling HIV viral mechanics. Mathematical Biosciences & Engineering, 2015, 12 (5) : 937-964. doi: 10.3934/mbe.2015.12.937 |
[4] |
Alex Capaldi, Samuel Behrend, Benjamin Berman, Jason Smith, Justin Wright, Alun L. Lloyd. Parameter estimation and uncertainty quantification for an epidemic model. Mathematical Biosciences & Engineering, 2012, 9 (3) : 553-576. doi: 10.3934/mbe.2012.9.553 |
[5] |
Ryan Bennink, Ajay Jasra, Kody J. H. Law, Pavel Lougovski. Estimation and uncertainty quantification for the output from quantum simulators. Foundations of Data Science, 2019, 1 (2) : 157-176. doi: 10.3934/fods.2019007 |
[6] |
Richard Archibald, Feng Bao, Yanzhao Cao, He Zhang. A backward SDE method for uncertainty quantification in deep learning. Discrete and Continuous Dynamical Systems - S, 2022 doi: 10.3934/dcdss.2022062 |
[7] |
Michael Herty, Elisa Iacomini. Uncertainty quantification in hierarchical vehicular flow models. Kinetic and Related Models, 2022, 15 (2) : 239-256. doi: 10.3934/krm.2022006 |
[8] |
Giuseppe Bianchi, Lorenzo Bracciale, Keren Censor-Hillel, Andrea Lincoln, Muriel Médard. The one-out-of-k retrieval problem and linear network coding. Advances in Mathematics of Communications, 2016, 10 (1) : 95-112. doi: 10.3934/amc.2016.10.95 |
[9] |
Yu Chen, Zixian Cui, Shihan Di, Peibiao Zhao. Capital asset pricing model under distribution uncertainty. Journal of Industrial and Management Optimization, 2021 doi: 10.3934/jimo.2021113 |
[10] |
Jianfeng Feng, Mariya Shcherbina, Brunello Tirozzi. Stability of the dynamics of an asymmetric neural network. Communications on Pure and Applied Analysis, 2009, 8 (2) : 655-671. doi: 10.3934/cpaa.2009.8.655 |
[11] |
Ning Zhang, Qiang Wu. Online learning for supervised dimension reduction. Mathematical Foundations of Computing, 2019, 2 (2) : 95-106. doi: 10.3934/mfc.2019008 |
[12] |
Lyudmila Grigoryeva, Juan-Pablo Ortega. Dimension reduction in recurrent networks by canonicalization. Journal of Geometric Mechanics, 2021, 13 (4) : 647-677. doi: 10.3934/jgm.2021028 |
[13] |
R.L. Sheu, M.J. Ting, I.L. Wang. Maximum flow problem in the distribution network. Journal of Industrial and Management Optimization, 2006, 2 (3) : 237-254. doi: 10.3934/jimo.2006.2.237 |
[14] |
Parker Childs, James P. Keener. Slow manifold reduction of a stochastic chemical reaction: Exploring Keizer's paradox. Discrete and Continuous Dynamical Systems - B, 2012, 17 (6) : 1775-1794. doi: 10.3934/dcdsb.2012.17.1775 |
[15] |
José Miguel Pasini, Tuhin Sahai. Polynomial chaos based uncertainty quantification in Hamiltonian, multi-time scale, and chaotic systems. Journal of Computational Dynamics, 2014, 1 (2) : 357-375. doi: 10.3934/jcd.2014.1.357 |
[16] |
Georgios I. Papayiannis. Robust policy selection and harvest risk quantification for natural resources management under model uncertainty. Journal of Dynamics and Games, 2022, 9 (2) : 203-217. doi: 10.3934/jdg.2022004 |
[17] |
Xueli Bai, Suying Liu. A new criterion to a two-chemical substances chemotaxis system with critical dimension. Discrete and Continuous Dynamical Systems - B, 2018, 23 (9) : 3717-3721. doi: 10.3934/dcdsb.2018074 |
[18] |
Sean D. Lawley, Janet A. Best, Michael C. Reed. Neurotransmitter concentrations in the presence of neural switching in one dimension. Discrete and Continuous Dynamical Systems - B, 2016, 21 (7) : 2255-2273. doi: 10.3934/dcdsb.2016046 |
[19] |
Ndolane Sene. Fractional input stability and its application to neural network. Discrete and Continuous Dynamical Systems - S, 2020, 13 (3) : 853-865. doi: 10.3934/dcdss.2020049 |
[20] |
Ying Sue Huang, Chai Wah Wu. Stability of cellular neural network with small delays. Conference Publications, 2005, 2005 (Special) : 420-426. doi: 10.3934/proc.2005.2005.420 |
2021 Impact Factor: 1.865
Tools
Metrics
Other articles
by authors
[Back to Top]