
-
Previous Article
Effective Mori-Zwanzig equation for the reduced-order modeling of stochastic systems
- DCDS-S Home
- This Issue
-
Next Article
An out-of-distribution-aware autoencoder model for reduced chemical kinetics
Augmented Gaussian random field: Theory and computation
1. | Department of Mathematics, Purdue University, West Lafayette, IN 47907, USA |
2. | Department of Industrial and Systems Engineering, Lehigh University, Bethlehem, PA 18015, USA |
3. | School of Mechanical Engineering, Purdue University, West Lafayette, IN 47907, USA |
We propose the novel augmented Gaussian random field (AGRF), which is a universal framework incorporating the data of observable and derivatives of any order. Rigorous theory is established. We prove that under certain conditions, the observable and its derivatives of any order are governed by a single Gaussian random field, which is the aforementioned AGRF. As a corollary, the statement "the derivative of a Gaussian process remains a Gaussian process" is validated, since the derivative is represented by a part of the AGRF. Moreover, a computational method corresponding to the universal AGRF framework is constructed. Both noiseless and noisy scenarios are considered. Formulas of the posterior distributions are deduced in a nice closed form. A significant advantage of our computational method is that the universal AGRF framework provides a natural way to incorporate arbitrary order derivatives and deal with missing data. We use four numerical examples to demonstrate the effectiveness of the computational method. The numerical examples are composite function, damped harmonic oscillator, Korteweg-De Vries equation, and Burgers' equation.
References:
[1] |
P. Abrahamsen and N. regnesentral, A Review of Gaussian Random Fields and Correlation Functions, Norsk Regnesentral/Norwegian Computing Center, 1997. |
[2] |
R. J. Adler, The Geometry of Random Fields, Society for Industrial and Applied Mathematics, 2010.
doi: 10.1137/1.9780898718980.ch1. |
[3] |
O. A. Chkrebtii, D. A. Campbell, B. Calderhead and M. A. Girolami,
Bayesian solution uncertainty quantification for differential equations, Bayesian Analysis, 11 (2016), 1239-1267.
doi: 10.1214/16-BA1017. |
[4] |
H.-S. Chung and J. Alonso, Using gradients to construct cokriging approximation models for high-dimensional design optimization problems, in 40th AIAA Aerospace Sciences Meeting & Exhibit, American Institute of Aeronautics and Astronautics, 2002.
doi: 10.2514/6.2002-317. |
[5] |
J. Cockayne, C. J. Oates, T. J. Sullivan and M. Girolami, Bayesian probabilistic numerical methods, SIAM Review, 61 (2019), 756–789.
doi: 10.1137/17M1139357. |
[6] |
H. Cramér and M. R. Leadbetter, Stationary and Related Stochastic Processes: Sample Function Properties and their Applications, Dover Books on Mathematics, Dover Publications, Inc., Mineola, NY, 2004. |
[7] |
S. Da Veiga and A. Marrel,
Gaussian process modeling with inequality constraints, Annales de la Faculté Des Sciences de Toulouse Mathématiques, 21 (2012), 529-555.
doi: 10.5802/afst.1344. |
[8] |
Y. Deng, G. Lin and X. Yang,
Multifidelity data fusion via gradient-enhanced Gaussian process regression, Communications in Computational Physics, 28 (2020), 1812-1837.
doi: 10.4208/cicp.OA-2020-0151. |
[9] |
R. M. Dudley, Real Analysis and Probability, No. 74 in Cambridge Studies in Advanced Mathematics, Cambridge University Press, 2002.
doi: 10.1017/CBO9780511755347. |
[10] |
A. I. Forrester, A. Sóbester and A. J. Keane,
Multi-fidelity optimization via surrogate modelling, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 463 (2007), 3251-3269.
doi: 10.1098/rspa.2007.1900. |
[11] |
P. Hennig, M. A. Osborne and M. Girolami, Probabilistic numerics and uncertainty in computations, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 471 (2015), 20150142.
doi: 10.1098/rspa.2015.0142. |
[12] |
M. C. Kennedy and A. O'Hagan,
Predicting the output from a complex computer code when fast approximations are available, Biometrika, 87 (2000), 1-13.
doi: 10.1093/biomet/87.1.1. |
[13] |
P. K. Kitanidis, Introduction to Geostatistics: Applications to Hydrogeology, Cambridge University Press, 1997.
doi: 10.1017/CBO9780511626166.![]() ![]() |
[14] |
L. Koralov and Y. G. Sinai, Theory of Probability and Random Processes, Universitext, Springer Berlin Heidelberg, 2007.
doi: 10.1007/978-3-540-68829-7. |
[15] |
L. Laurent, R. Le Riche, B. Soulier and P.-A. Boucard,
An overview of gradient-enhanced metamodels with applications, Archives of Computational Methods in Engineering, 26 (2019), 61-106.
doi: 10.1007/s11831-017-9226-3. |
[16] |
L. Le Gratiet and J. Garnier,
Recursive co-kriging model for design of computer experiments with multiple levels of fidelity, International Journal for Uncertainty Quantification, 4 (2014), 365-386.
doi: 10.1615/Int.J.UncertaintyQuantification.2014006914. |
[17] |
L. Lin and D. B. Dunson,
Bayesian monotone regression using Gaussian process projection, Biometrika, 101 (2014), 303-317.
doi: 10.1093/biomet/ast063. |
[18] |
H. Liu, Y.-S. Ong, X. Shen and J. Cai,
When Gaussian process meets big data: A review of scalable GPs, IEEE Transactions on Neural Networks and Learning Systems, 31 (2020), 4405-4423.
doi: 10.1109/TNNLS.2019.2957109. |
[19] |
A. F. López-Lopera, F. Bachoc, N. Durrande and O. Roustant, Finite-dimensional Gaussian approximation with linear inequality constraints, SIAM/ASA Journal on Uncertainty Quantification, 6 (2018), 1224–1255.
doi: 10.1137/17M1153157. |
[20] |
M. D. Morris, T. J. Mitchell and D. Ylvisaker,
Bayesian design and analysis of computer experiments: Use of derivatives in surface prediction, Technometrics, 35 (1993), 243-255.
doi: 10.1080/00401706.1993.10485320. |
[21] |
A. Pensoneault, X. Yang and X. Zhu,
Nonnegativity-enforced Gaussian process regression, Theoretical and Applied Mechanics Letters, 10 (2020), 182-187.
doi: 10.1016/j.taml.2020.01.036. |
[22] |
P. Perdikaris, M. Raissi, A. Damianou, N. D. Lawrence and G. E. Karniadakis, Nonlinear information fusion algorithms for data-efficient multi-fidelity modelling, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 473 (2017), 20160751.
doi: 10.1098/rspa.2016.0751. |
[23] |
M. Raissi, P. Perdikaris and G. E. Karniadakis,
Machine learning of linear differential equations using Gaussian processes, Journal of Computational Physics, 348 (2017), 683-693.
doi: 10.1016/j.jcp.2017.07.050. |
[24] |
M. Raissi, P. Perdikaris and G. E. Karniadakis, Numerical Gaussian processes for time-dependent and nonlinear partial differential equations, SIAM Journal on Scientific Computing, 40 (2018), A172–A198.
doi: 10.1137/17M1120762. |
[25] |
C. E. Rasmussen and C. K. I. Williams, Gaussian Processes for Machine Learning, Adaptive computation and machine learning, MIT Press, 2006. |
[26] |
J. Sacks, W. J. Welch, T. J. Mitchell and H. P. Wynn,
Design and analysis of computer experiments, Statistical Science, 4 (1989), 409-423.
|
[27] |
M. Schober, D. Duvenaud and P. Hennig, Probabilistic ODE solvers with runge-kutta means, arXiv: 1406.2582, [cs, math, stat], 2014. |
[28] |
E. Solak, R. Murray-smith, W. E. Leithead, D. J. Leith and C. E. Rasmussen, Derivative observations in Gaussian process models of dynamic systems, in Advances in Neural Information Processing Systems 15 (S. Becker, S. Thrun, and K. Obermayer, eds.), 1057–1064,
MIT Press, 2003. |
[29] |
S. Ulaganathan, I. Couckuyt, F. Ferranti, E. Laermans and T. Dhaene,
Performance study of multi-fidelity gradient enhanced kriging, Structural and Multidisciplinary Optimization, 51 (2015), 1017-1033.
doi: 10.1007/s00158-014-1192-x. |
[30] |
X. Yang, D. Barajas-Solano, G. Tartakovsky and A. M. Tartakovsky,
Physics-informed CoKriging: A Gaussian-process-regression-based multifidelity method for data-model convergence, Journal of Computational Physics, 395 (2019), 410-431.
doi: 10.1016/j.jcp.2019.06.041. |
[31] |
X. Yang, G. Tartakovsky and A. Tartakovsky, Physics-informed kriging: A physics-informed Gaussian process regression method for data-model convergence, arXiv: 1809.03461, [cs, stat], 2018. |
[32] |
X. Yang, X. Zhu and J. Li, When bifidelity meets CoKriging: An efficient physics-informed MultiFidelity method, SIAM Journal on Scientific Computing, 42 (2020), A220–A249.
doi: 10.1137/18M1231353. |
show all references
References:
[1] |
P. Abrahamsen and N. regnesentral, A Review of Gaussian Random Fields and Correlation Functions, Norsk Regnesentral/Norwegian Computing Center, 1997. |
[2] |
R. J. Adler, The Geometry of Random Fields, Society for Industrial and Applied Mathematics, 2010.
doi: 10.1137/1.9780898718980.ch1. |
[3] |
O. A. Chkrebtii, D. A. Campbell, B. Calderhead and M. A. Girolami,
Bayesian solution uncertainty quantification for differential equations, Bayesian Analysis, 11 (2016), 1239-1267.
doi: 10.1214/16-BA1017. |
[4] |
H.-S. Chung and J. Alonso, Using gradients to construct cokriging approximation models for high-dimensional design optimization problems, in 40th AIAA Aerospace Sciences Meeting & Exhibit, American Institute of Aeronautics and Astronautics, 2002.
doi: 10.2514/6.2002-317. |
[5] |
J. Cockayne, C. J. Oates, T. J. Sullivan and M. Girolami, Bayesian probabilistic numerical methods, SIAM Review, 61 (2019), 756–789.
doi: 10.1137/17M1139357. |
[6] |
H. Cramér and M. R. Leadbetter, Stationary and Related Stochastic Processes: Sample Function Properties and their Applications, Dover Books on Mathematics, Dover Publications, Inc., Mineola, NY, 2004. |
[7] |
S. Da Veiga and A. Marrel,
Gaussian process modeling with inequality constraints, Annales de la Faculté Des Sciences de Toulouse Mathématiques, 21 (2012), 529-555.
doi: 10.5802/afst.1344. |
[8] |
Y. Deng, G. Lin and X. Yang,
Multifidelity data fusion via gradient-enhanced Gaussian process regression, Communications in Computational Physics, 28 (2020), 1812-1837.
doi: 10.4208/cicp.OA-2020-0151. |
[9] |
R. M. Dudley, Real Analysis and Probability, No. 74 in Cambridge Studies in Advanced Mathematics, Cambridge University Press, 2002.
doi: 10.1017/CBO9780511755347. |
[10] |
A. I. Forrester, A. Sóbester and A. J. Keane,
Multi-fidelity optimization via surrogate modelling, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 463 (2007), 3251-3269.
doi: 10.1098/rspa.2007.1900. |
[11] |
P. Hennig, M. A. Osborne and M. Girolami, Probabilistic numerics and uncertainty in computations, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 471 (2015), 20150142.
doi: 10.1098/rspa.2015.0142. |
[12] |
M. C. Kennedy and A. O'Hagan,
Predicting the output from a complex computer code when fast approximations are available, Biometrika, 87 (2000), 1-13.
doi: 10.1093/biomet/87.1.1. |
[13] |
P. K. Kitanidis, Introduction to Geostatistics: Applications to Hydrogeology, Cambridge University Press, 1997.
doi: 10.1017/CBO9780511626166.![]() ![]() |
[14] |
L. Koralov and Y. G. Sinai, Theory of Probability and Random Processes, Universitext, Springer Berlin Heidelberg, 2007.
doi: 10.1007/978-3-540-68829-7. |
[15] |
L. Laurent, R. Le Riche, B. Soulier and P.-A. Boucard,
An overview of gradient-enhanced metamodels with applications, Archives of Computational Methods in Engineering, 26 (2019), 61-106.
doi: 10.1007/s11831-017-9226-3. |
[16] |
L. Le Gratiet and J. Garnier,
Recursive co-kriging model for design of computer experiments with multiple levels of fidelity, International Journal for Uncertainty Quantification, 4 (2014), 365-386.
doi: 10.1615/Int.J.UncertaintyQuantification.2014006914. |
[17] |
L. Lin and D. B. Dunson,
Bayesian monotone regression using Gaussian process projection, Biometrika, 101 (2014), 303-317.
doi: 10.1093/biomet/ast063. |
[18] |
H. Liu, Y.-S. Ong, X. Shen and J. Cai,
When Gaussian process meets big data: A review of scalable GPs, IEEE Transactions on Neural Networks and Learning Systems, 31 (2020), 4405-4423.
doi: 10.1109/TNNLS.2019.2957109. |
[19] |
A. F. López-Lopera, F. Bachoc, N. Durrande and O. Roustant, Finite-dimensional Gaussian approximation with linear inequality constraints, SIAM/ASA Journal on Uncertainty Quantification, 6 (2018), 1224–1255.
doi: 10.1137/17M1153157. |
[20] |
M. D. Morris, T. J. Mitchell and D. Ylvisaker,
Bayesian design and analysis of computer experiments: Use of derivatives in surface prediction, Technometrics, 35 (1993), 243-255.
doi: 10.1080/00401706.1993.10485320. |
[21] |
A. Pensoneault, X. Yang and X. Zhu,
Nonnegativity-enforced Gaussian process regression, Theoretical and Applied Mechanics Letters, 10 (2020), 182-187.
doi: 10.1016/j.taml.2020.01.036. |
[22] |
P. Perdikaris, M. Raissi, A. Damianou, N. D. Lawrence and G. E. Karniadakis, Nonlinear information fusion algorithms for data-efficient multi-fidelity modelling, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 473 (2017), 20160751.
doi: 10.1098/rspa.2016.0751. |
[23] |
M. Raissi, P. Perdikaris and G. E. Karniadakis,
Machine learning of linear differential equations using Gaussian processes, Journal of Computational Physics, 348 (2017), 683-693.
doi: 10.1016/j.jcp.2017.07.050. |
[24] |
M. Raissi, P. Perdikaris and G. E. Karniadakis, Numerical Gaussian processes for time-dependent and nonlinear partial differential equations, SIAM Journal on Scientific Computing, 40 (2018), A172–A198.
doi: 10.1137/17M1120762. |
[25] |
C. E. Rasmussen and C. K. I. Williams, Gaussian Processes for Machine Learning, Adaptive computation and machine learning, MIT Press, 2006. |
[26] |
J. Sacks, W. J. Welch, T. J. Mitchell and H. P. Wynn,
Design and analysis of computer experiments, Statistical Science, 4 (1989), 409-423.
|
[27] |
M. Schober, D. Duvenaud and P. Hennig, Probabilistic ODE solvers with runge-kutta means, arXiv: 1406.2582, [cs, math, stat], 2014. |
[28] |
E. Solak, R. Murray-smith, W. E. Leithead, D. J. Leith and C. E. Rasmussen, Derivative observations in Gaussian process models of dynamic systems, in Advances in Neural Information Processing Systems 15 (S. Becker, S. Thrun, and K. Obermayer, eds.), 1057–1064,
MIT Press, 2003. |
[29] |
S. Ulaganathan, I. Couckuyt, F. Ferranti, E. Laermans and T. Dhaene,
Performance study of multi-fidelity gradient enhanced kriging, Structural and Multidisciplinary Optimization, 51 (2015), 1017-1033.
doi: 10.1007/s00158-014-1192-x. |
[30] |
X. Yang, D. Barajas-Solano, G. Tartakovsky and A. M. Tartakovsky,
Physics-informed CoKriging: A Gaussian-process-regression-based multifidelity method for data-model convergence, Journal of Computational Physics, 395 (2019), 410-431.
doi: 10.1016/j.jcp.2019.06.041. |
[31] |
X. Yang, G. Tartakovsky and A. Tartakovsky, Physics-informed kriging: A physics-informed Gaussian process regression method for data-model convergence, arXiv: 1809.03461, [cs, stat], 2018. |
[32] |
X. Yang, X. Zhu and J. Li, When bifidelity meets CoKriging: An efficient physics-informed MultiFidelity method, SIAM Journal on Scientific Computing, 42 (2020), A220–A249.
doi: 10.1137/18M1231353. |





[1] |
Johnathan M. Bardsley. Gaussian Markov random field priors for inverse problems. Inverse Problems and Imaging, 2013, 7 (2) : 397-416. doi: 10.3934/ipi.2013.7.397 |
[2] |
John Maclean, Elaine T. Spiller. A surrogate-based approach to nonlinear, non-Gaussian joint state-parameter data assimilation. Foundations of Data Science, 2021, 3 (3) : 589-614. doi: 10.3934/fods.2021019 |
[3] |
Rajae Aboulaϊch, Amel Ben Abda, Moez Kallel. Missing boundary data reconstruction via an approximate optimal control. Inverse Problems and Imaging, 2008, 2 (4) : 411-426. doi: 10.3934/ipi.2008.2.411 |
[4] |
Peter Giesl, Boumediene Hamzi, Martin Rasmussen, Kevin Webster. Approximation of Lyapunov functions from noisy data. Journal of Computational Dynamics, 2020, 7 (1) : 57-81. doi: 10.3934/jcd.2020003 |
[5] |
Alfred K. Louis. Diffusion reconstruction from very noisy tomographic data. Inverse Problems and Imaging, 2010, 4 (4) : 675-683. doi: 10.3934/ipi.2010.4.675 |
[6] |
Yan Wang, Lei Wang, Yanxiang Zhao, Aimin Song, Yanping Ma. A stochastic model for microbial fermentation process under Gaussian white noise environment. Numerical Algebra, Control and Optimization, 2015, 5 (4) : 381-392. doi: 10.3934/naco.2015.5.381 |
[7] |
Hongjun Gao, Fei Liang. On the stochastic beam equation driven by a Non-Gaussian Lévy process. Discrete and Continuous Dynamical Systems - B, 2014, 19 (4) : 1027-1045. doi: 10.3934/dcdsb.2014.19.1027 |
[8] |
Yohei Fujishima. On the effect of higher order derivatives of initial data on the blow-up set for a semilinear heat equation. Communications on Pure and Applied Analysis, 2018, 17 (2) : 449-475. doi: 10.3934/cpaa.2018025 |
[9] |
Max-Olivier Hongler. Mean-field games and swarms dynamics in Gaussian and non-Gaussian environments. Journal of Dynamics and Games, 2020, 7 (1) : 1-20. doi: 10.3934/jdg.2020001 |
[10] |
Justyna Jarczyk, Witold Jarczyk. Gaussian iterative algorithm and integrated automorphism equation for random means. Discrete and Continuous Dynamical Systems, 2020, 40 (12) : 6837-6844. doi: 10.3934/dcds.2020135 |
[11] |
Yuta Tanoue. Improved Hoeffding inequality for dependent bounded or sub-Gaussian random variables. Probability, Uncertainty and Quantitative Risk, 2021, 6 (1) : 53-60. doi: 10.3934/puqr.2021003 |
[12] |
Zhenghong Qiu, Jianhui Huang, Tinghan Xie. Linear-Quadratic-Gaussian mean-field controls of social optima. Mathematical Control and Related Fields, 2021 doi: 10.3934/mcrf.2021047 |
[13] |
G. Caginalp, Emre Esenturk. Anisotropic phase field equations of arbitrary order. Discrete and Continuous Dynamical Systems - S, 2011, 4 (2) : 311-350. doi: 10.3934/dcdss.2011.4.311 |
[14] |
Habib Ammari, Josselin Garnier, Vincent Jugnon. Detection, reconstruction, and characterization algorithms from noisy data in multistatic wave imaging. Discrete and Continuous Dynamical Systems - S, 2015, 8 (3) : 389-417. doi: 10.3934/dcdss.2015.8.389 |
[15] |
Xiaoman Liu, Jijun Liu. Image restoration from noisy incomplete frequency data by alternative iteration scheme. Inverse Problems and Imaging, 2020, 14 (4) : 583-606. doi: 10.3934/ipi.2020027 |
[16] |
Müge Acar, Refail Kasimbeyli. A polyhedral conic functions based classification method for noisy data. Journal of Industrial and Management Optimization, 2021, 17 (6) : 3493-3508. doi: 10.3934/jimo.2020129 |
[17] |
Xuping Xie, Feng Bao, Thomas Maier, Clayton Webster. Analytic continuation of noisy data using Adams Bashforth residual neural network. Discrete and Continuous Dynamical Systems - S, 2022, 15 (4) : 877-892. doi: 10.3934/dcdss.2021088 |
[18] |
Guangsheng Wei, Hong-Kun Xu. On the missing bound state data of inverse spectral-scattering problems on the half-line. Inverse Problems and Imaging, 2015, 9 (1) : 239-255. doi: 10.3934/ipi.2015.9.239 |
[19] |
Bruno Sixou, Cyril Mory. Kullback-Leibler residual and regularization for inverse problems with noisy data and noisy operator. Inverse Problems and Imaging, 2019, 13 (5) : 1113-1137. doi: 10.3934/ipi.2019050 |
[20] |
Jiang Xie, Junfu Xu, Celine Nie, Qing Nie. Machine learning of swimming data via wisdom of crowd and regression analysis. Mathematical Biosciences & Engineering, 2017, 14 (2) : 511-527. doi: 10.3934/mbe.2017031 |
2021 Impact Factor: 1.865
Tools
Metrics
Other articles
by authors
[Back to Top]