Performance Increases in Physics-Informed Machine Learning using Hypercomplex-based Automatic Differentiation (HYPAD)

Abstract

Physics-Informed Machine Learning (PIML) integrates physical laws into neural network training to solve complex problems governed by differential equations. A critical component in PIML is computing derivatives of models to ensure compliance with the governing equations. A common approach is to use automatic differentiation (AD) methods available in machine learning frameworks such as PyTorch, which is accurate but not always the most efficient approach. An alternative AD method is demonstrated here in the context of physics-regularized genetic programming with symbolic regression (PR-GPSR). This work introduces Hypercomplex-based Automatic Differentiation (HYPAD), which utilizes the Order Truncated Imaginary (OTI) library to efficiently compute second- and fourth-order derivatives in PR-GPSR applications via BINGO, an open-source symbolic regression library. Numerical experiments demonstrate that HYPAD achieves up to a \(123\times\) runtime improvement over PyTorch implementations, with both approaches recovering the correct solutions in the demonstrated PR-GPSR problems. HYPAD evaluates functions by propagating OTI hypercomplex numbers through a command array representation, enabling computation of arbitrary-order derivatives. The approach may also be applicable beyond symbolic regression, motivating further investigation in other PIML and AD applications.