$\lambda$-JEPA Spectral Anti-Collapse Regularization for Self-Supervised Learning
This work introduces SACReg, a spectral anti-collapse regularizer motivated by an analysis of $\lambda$-balance, which captures the relative scale of weight matrices across layers, and proposes $\lambda$-JEPA, which improves over LeJEPA and VISReg on ImageNet-1k classification and in average linear-probe transfer perfo...
Berker Demirel, C. Dominé, Valentino Maiorca et al.
· 0 citations