Preventing Model Collapse: A Fisher-Rao Perspective on the Dynamics of Training with Synthetic Data
It is shown that the effective required data ratio to prevent model collapse is different than previously implied, and quantitative contraction and invariance bounds that are stable and do not become trivial as the dimensions increase are derived.
Matteo Marchi, João Pedro Silvestre, Bahman Gharesifard et al.
· 0 citations