Open-ended reinforcement learning often relies on rubric-based rewards for tasks without directly verifiable answers. Yet the policy and reward system form a dynamic feedback loop: as the policy optimizes the current reward, an initially useful reward system may become unreliable due to reward hacking or reduced response discriminability. The reward system should therefore evolve rather than remain fixed during training. Existing dynamic-rubric methods adapt evaluation criteria, but reward failures can also arise from scoring mechanisms or signal composition. We introduce EvoRS, a self-evolving RL framework that evolves the reward system from on-policy experience, representing it as an executable Reward-DAG. Specifically, an agentic designer updates this system from on-policy rollouts and reward traces to maintain train-time reliability. Across writing and roleplay, EvoRS achieves the best quality under all three judges, outperforming the policy by \(2.107\) and \(4.767\) points, respectively, while reducing reward hacking and coverage failures and preserving reward informativeness. Ablations confirm that a comprehensive fixed reward system cannot remain reliable in open-ended tasks and must evolve throughout training.
Weiyuan Li, Aili Chen, Xin-Tao Wang et al.· 0 citations
CAST (Credit Assignment from Solver Teachers), which converts value changes in a game solver's state value into solver advantages and injects them into RLVR as turn-level signals and achieves the highest average zero-shot performance on ALFWorld and WebShop.
Yu Wang, Yi-Kai Zhang, Wentao Shi et al.· arXiv.org· 0 citations
HarnessLens is introduced, a budget-aware framework for automated harness evolution that jointly explores the task space and user-configurable components, derives candidate modifications from execution trajectories, and selectively verifies each candidate on behavior-relevant tasks using an attributable-evidence gate.
Jingheng Xu, Yi-Kai Zhang, Aiden Chen et al.· 1 citation
It is argued that the next leap in AI4Math systems requires a decisive shift from predefined problem-solvers to research agents that can address frontier mathematical challenges with rigorous formal mathematical reasoning, highlighting core limitations of existing systems in serving as mathematical research agents.
E. Jiang, Xiao Liang, Yikai Zhang et al.· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
