Predictive Divergence Masks for LLM RL
The predictive divergence mask is proposed, which asks whether the next policy-gradient step will increase or decrease the same divergence used by the trust region, and the resulting masks improve RL training across model scales and precision settings.