SoftmaxGRPO: Learning to Reason using Softmax Advantage Group Estimation
Softmax Advantage Group Estimation (SoftmaxGRPO), a drop-in alternative that replaces z-score-normalized group advantages with temperature-scaled softmax advantages, keeping weights bounded regardless of prompt difficulty, is introduced.