Where to Start, How to Move: Analyzing Curricula under GRPO for Long-Horizon LLM Agents
Group Relative Policy Optimization (GRPO) provides no reward-relative learning signal when all rollouts receive identical rewards. In long-horizon tasks that require multiple interactions, rewards are given only upon successful task completion, making all-fail groups common early in training and limiting opportunities...