Zhiqiu Yu

Unveiling Implicit Advantage Symmetry: Why GRPO Struggles with Exploration and Difficulty Adaptation

This work identifies an implicit advantage symmetry inherent in Group Relative Advantage Estimation (GRAE) as a structural property that provides a new perspective for understanding bottlenecks in reinforcement learning, and proposes Asymmetric GRAE (A-GRAE), which dynamically modulates exploration incentives and sampl...