Beyond the Manifold Hypothesis: Hybrid Spectral Parameterizations for Flow Matching

Abstract

Flow matching and diffusion can be trained to predict different quantities, most commonly the data $x_1$, the source noise $x_0$, or the velocity~$v$. Although theoretically equivalent, these can lead to substantially different performances. We identify two main drivers for these differences: the source--data signal-to-noise ratio, and the information bottleneck induced by the neural architecture. We show that, beyond intrinsic data dimension, the factor affecting the optimal parametrization the most is a certain signal-to-noise ratio in each data covariance direction. From this analysis, we introduce new \emph{spectral hybrid} parameterizations that adapt across time and data covariance directions; we show that these are optimal for Gaussian data. We also show that architecture-induced compression changes which parameterization is easier to learn, with $v$-prediction being more sensitive to discarded directions than $x_1$-prediction. Experiments across architectures and source scales show that our spectral parameterizations are robust across regimes, can substantially accelerate optimization, while incurring essentially no additional training cost compared with standard parameterizations.