Not All Preferences Deserve Gradients: Understanding Gradient Utility in Offline Reasoning Alignment
SAGE (Stability-Aware Gradient Efficiency), which maintains difficulty-stratified candidate pools refreshed during training and selects pairs within each pool by a forward-pass signal-to-curvature score, outperforms full-data and size-matched baselines while producing substantially smoother optimization trajectories.
Hui Wu, Hengyi Cai, Jin-Man Zhao et al.
· 0 citations