Multi-output Orthogonal Gaussian Processes for Noisy Simulator Outputs
TL;DR
A definition of orthogonality for multi-output Gaussian processes, a construction that enforces orthogonality by conditioning each covariance function, and a structured likelihood formulation that significantly improves computational scaling are introduced.
Abstract
Computer simulations can model physical processes but are often too expensive to produce enough runs for calibration, sensitivity analysis, prediction, and uncertainty quantification. As a result, statistical surrogates are frequently used as cheaper alternatives that can be trained on a small number of simulation runs. Gaussian processes (GP) are well-suited as surrogates, as they provide flexible, nonlinear regression and closed-form uncertainty quantification. However, standard GPs are insufficient replacements for more complex simulators, such as stochastic or multi-output simulators. In addition, typical GPs also struggle to separate fitted regression coefficients from residual process variation. We introduce multi-output orthogonal Gaussian process (MOOGP) to tackle the problems mentioned above. This paper has three main contributions: (i) a definition of orthogonality for multi-output Gaussian processes, (ii) a construction that enforces orthogonality by conditioning each covariance function, and (iii) a structured likelihood formulation that significantly improves computational scaling. In a trend-recovery illustrative example, MOOGP recovers the true trend while the non-orthogonalized counterpart fails to do so, to the extent it reverses the sign of the trend. Numerical experiments and an application in heavy-ion collision simulations further demonstrate the interpretability and predictive advantage of MOOGP.