Learning Differential Equations From Numerically Integrated Artificial Neural Networks
TL;DR
An approach is presented which proposes the embedding of a neural network based architecture as a substitute for the state function of an ODE into established Runge–Kutta based integration method.
Abstract
For numerical investigation of dynamical systems, the formulation of the corresponding ordinary differential equations (ODE) based on physical principles is usually the first and most crucial step. However, if the underlying physics is not fully understood or the required expert knowledge for modeling is missing, setting up these differential equations fail. Sometimes, running either real‐world experiments or black‐box simulations with commercial of‐the‐shelf software are the only ways of system exploration, which can be time‐consuming and/or expensive. In such cases, based on the gathered data, a surrogate model for the ODE may be set up and trained with the goal of later substituting the missing differential equation for cheaper numerical investigations. In this paper, an approach is presented which proposes the embedding of a neural network based architecture as a substitute for the state function of an ODE into established Runge–Kutta based integration method. For training the surrogate, the numerical integration method solves an initial value problem to map initial conditions onto target system states for comparison with training data. The corresponding loss is then backpropagated through the model graph spanned by the numerical integration scheme to update the adjustable weights of the neural network for minimizing the loss of the mapping. The optimized surrogate state function may finally be treated as a substitute for the differential equation under investigation.