This thesis advances the training and scalability of NCDEs through three complementary contributions, building on neural rough differential equations, which reduce the time per training step for an NCDE by up to three orders of magnitude while achieving state-of-the-art performance across diverse time series benchmarks.
Abstract
Many real-world systems evolve continuously, yet most machine learning models interpret time series as discrete sequences. Continuous-time approaches instead treat time series as samples from an underlying input path, a formulation that naturally accommodates irregularly sampled or oversampled data. Among these, Neural Controlled Differential Equations (NCDEs) are a maximally expressive class of models that parametrise a vector field using a neural network and evolve their hidden state by solving a dynamical system driven by the input path. NCDEs typically use a non-linear vector field, so their expressive power and continuous-time flexibility come at the cost of a forward pass that is both computationally expensive and inherently sequential, limiting their scalability and practical applicability. This thesis advances the training and scalability of NCDEs through three complementary contributions. First, building on neural rough differential equations, Log-NCDEs apply the Log-ODE method to efficiently approximate an NCDE's solution during training, improving both computational speed and empirical performance. Second, Linear NCDEs replace the non-linear vector field with a linear one, enabling closed-form solutions and parallel-in-time computation without sacrificing theoretical expressivity. Third, Structured Linear NCDEs use structured linear vector fields to further enhance efficiency while maintaining theoretical expressiveness and empirical performance. Collectively, these methods reduce the time per training step for an NCDE by up to three orders of magnitude while achieving state-of-the-art performance across diverse time series benchmarks.
We present a machine learning framework for identifying sparse, interpretable models of dynamical systems directly from time-series data. Our approach parameterizes the underlying vector field using a neural architecture and trains it by minimizing a multi-step prediction loss over a finite horizon. To ensure numerical tractability, we optimize a mean absolute error objective averaged across prediction steps, and progressively increase the horizon during training. A key feature of this formulation is that it enforces consistency under repeated composition of the learned dynamics. As a result, the identified models exhibit significantly improved stability compared with approaches based on one-step regression of the vector field. When combined with sparsity-promoting regularization, this leads to parsimonious models that generalize beyond the training data. We demonstrate accurate recovery of systems exhibiting a wide range of behaviors, including stable and unstable fixed points, periodic orbits, and chaotic attractors. For chaotic systems, while long-term trajectory prediction is inherently limited by sensitivity to initial conditions, we show that multi-step training yields models with accurate short-term dynamics and strong agreement in long-time statistical properties, including mean, variance, and Lyapunov exponents. Moreover, we establish theoretical bounds linking trajectory error to statistical accuracy, providing a step toward a principled explanation for this behavior.
One of the main objectives in control theory is to obtain a linear representation of inherently nonlinear systems in order to leverage the analytical and theoretical tools developed for linear systems. In this context, the Koopman operator has attracted increasing interest in recent years.Koopman operator theory provides a framework in which nonlinear dynamical systems are represented by a linear operator acting on an infinite-dimensional Hilbert space. Since such an infinite-dimensional representation is not numerically tractable, numerous finite-dimensional approximation methods have been proposed. These approaches typically rely on time-series data and include extended dynamic mode decomposition as well as deep learning–based variants. In this paper, we propose an original machine-learning-based approach for the synthesis of a fixed-dimensional Koopman approximant (lifting) of continuous-time nonlinear systems. A differential state-space representation of the system (as opposed to a recurrent state model) is assumed to be available through its vector field (f). The proposed encoder departs from conventional approaches in that it does not directly output the current latent state, but instead generates samples of the latent trajectory evaluated at user-defined time instants (temporal discretization). This formulation enables the integration into the learning process of Physical & Latent Continuous Losses, enforcing consistency between the physical dynamics and the Koopman dynamics, as well as Physical & Latent Boundary Losses, ensuring consistency with the prescribed initial conditions. In parallel, we introduce a structural stability constraint on the Koopman operator. The effectiveness of the proposed methodology is demonstrated through the analysis and simulation of two polynomial dynamical systems.
M. Zodros, A. Colotti, M. Yagoubi et al.· International Conference on...· 0 citations
Learning continuous-time representations of dynamical systems from observation data has emerged as a cornerstone of data-driven control and scientific machine learning. However, existing neural differential equations either treat external control inputs heuristically without providing strict structural guarantees, or enforce stability properties under the restrictive assumption of constant or vanishing inputs. This paper proposes the Input-Contraction Neural Differential Model (ICNDM), a novel deep learning framework that seamlessly incorporates time-varying control inputs while ensuring incremental exponential convergence via input-dependent contraction regularization. By leveraging an embedded input encoder and a parameterized metric network, the proposed architecture learns both the non-autonomous neural vector fields and a generalized Riemannian contraction metric simultaneously. We derive sufficient conditions for input-dependent contraction and formally establish an input-to-state contraction property under bounded external excitations. Extensive numerical evaluations on highly nonlinear chaotic oscillators and experimental data from a Permanent Magnet Synchronous Motor (PMSM) drive system demonstrate that ICNDM yields substantial reductions in long-horizon rollout errors and exhibits superior structural robustness against input perturbations compared with state-of-the-art neural differential benchmarks.
This work proposes Neural Kolmogorov Equations (NKEs), a deterministic, infinite-dimensional reformulation of Neural SDEs based on the Kolmogorov Forward equation, transforming the learning problem from modelling individual stochastic trajectories to modelling the evolution of probability densities.
Learning nonparametric systems of Ordinary Differential Equations (ODEs) from noisy data is challenging, especially when the system is input-dependent. Most current nonparametric approaches focus on autonomous systems, making them unable to capture the influence of external inputs. In this paper, we introduce a Bi-stage Gaussian Process (GP) framework for non-autonomous ODEs, capable of estimating system states and their derivatives directly from noisy measurements. The proposed method adopts a purely data-driven and nonparametric formulation, relying on Gaussian process regression and numerical integration without assuming explicit parametric system models or theoretical performance guarantees. The method is demonstrated on a scalar forced ODE with amplitudes <inline-formula> <tex-math notation="LaTeX">$A \in [{0.05, 2.5}]\pi $ </tex-math></inline-formula> and frequencies <inline-formula> <tex-math notation="LaTeX">$\omega \in [{0.1, 31.6}]$ </tex-math></inline-formula>, achieving state prediction errors below 2% for high signal-to-noise ratios (SNR = 1000) and derivative errors below 5% even for noisy measurements (SNR = 30). Furthermore, the approach is applied to a continuous stirred tank reactor (CSTR) system with inlet concentrations <inline-formula> <tex-math notation="LaTeX">$C_{A0}=1.0 2.0$ </tex-math></inline-formula> mol/m3 and flow rates <inline-formula> <tex-math notation="LaTeX">$F=0.01$ </tex-math></inline-formula> m3/s, successfully estimating reaction rates with relative errors below 4% across varying noise levels (SNR <inline-formula> <tex-math notation="LaTeX">$=100~30$ </tex-math></inline-formula>). Comparative results with non-parametric ODE (npODE), Gaussian Process ODE (GPODE) and continuous-time state-space neural network (CSNN) models demonstrate that the proposed Bi-stage GP achieves superior generalization performance under varying input conditions. The results demonstrate that the proposed method is robust, accurate, and capable of generalizing to unobserved inputs, providing a reliable alternative to classical ODE modeling in noisy and complex systems.
R. Fezai, Byanne Malluhi, N. Basha et al.· IEEE Access· 0 citations
Continuous-time neural models are attractive for identifying nonlinear systems, but a small one-step error can grow rapidly when a learned vector field is rolled out under inputs that differ from those used for training. This paper develops an adaptive stability-constrained neural differential equation (AS-NDE) for systems with measured controls and unmatched, unknown perturbations. The nominal vector field and a state--input-dependent Riemannian metric are learned jointly. Positive definiteness is enforced by construction, while a sampled differential inequality penalizes violations of a prescribed contraction rate. An incremental input-to-state bound is derived: the distance between two trajectories decays exponentially up to gains determined by differences in their controls and disturbances. The statement explicitly accounts for the time derivative of an input-dependent metric, a term that is easily omitted in heuristic stability regularizers. We give a reproducible evaluation protocol for a forced Duffing oscillator and a permanent-magnet synchronous motor (PMSM) model. Because no measured data or executed training runs accompany this draft, all numerical curves and tables are clearly identified as illustrative synthetic placeholders; their PGFPlots coordinates are embedded in the source for direct replacement. The resulting manuscript is intended as a technically consistent starting point, not as evidence of empirical superiority before the prescribed experiments are run.