A Sign Law for Entropy Drift under Sampler Mismatch in RLVR
It is proved that a constant advantage shift then changes the entropy of a softmax policy, at first order and in expectation, with a sign fixed by the sampler, as well as a covariance between advantages and the sampler's sharpness gap.
Zhi-Yuan Wang, Jacob Steeves, Chang Liu
· 1 citation