Lunyuan Hu

Robustness Emerges Early in Training Dynamics, but Is Not Preserved

This paper identifies a robustness fading phenomenon where shallow layers spontaneously develop robust representations and flat loss landscapes in early training, yet these properties are not preserved during standard convergence, and proposes a framework that performs strategic interventions on training dynamics to st...