Not All Preferences Deserve Gradients: Understanding Gradient Utility in Offline Reasoning Alignment
SAGE (Stability-Aware Gradient Efficiency), which maintains difficulty-stratified candidate pools refreshed during training and selects pairs within each pool by a forward-pass signal-to-curvature score, outperforms full-data and size-matched baselines while producing substantially smoother optimization trajectories.