A key strength of Proximal Policy Optimization (PPO) is its learned critic, which uses historical trajectories collected during reinforcement learning to estimate expected returns and reduce policy-gradient variance. However, we find that the critic is also a major source of instability in reinforcement learning for la...
Xuan-Yi Zhou, Qiu-Yang Mang, Huan-Zhi Mao et al.· 0 citations
Combee is proposed, a novel framework to scale parallel prompt learning for self-improving agents that speeds up learning and enables running many agents in parallel while learning from their aggregate traces without quality degradation.
This work defines the scaling inflection point as the per-session budget where marginal Elo gains match the independent-sampling reference, and proposes Elo-per-token analysis, which tracks the best solution found at each token budget and uses a Bradley-Terry model to aggregate within-task orderings into Elo ratings ac...
Kai-Yuan Liu, Qiu-Yang Mang, Bo-Fei Peng et al.· 3 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.