The sycophancy of language models, or their failure to push back against faulty user input, is now widely studied. However, if LMs do push back, is sycophancy solved? We show that LMs vary in how they present pushback, and thus vary in how effectively they correct user beliefs. We introduce a phrase-level, human-valida...
This work studies interventions at four levels of group-policy RL pipelines---data, rollout, reward, and advantage---and conducts extensive RL training on saturated reasoning data only, demonstrating that don't waste your saturated data: with the right strategies they can be recycled into useful RL training signals in...
An extensive evaluation of automatic harness evolution for LLM agents is conducted, comparing harness evolution with simple test-time scaling and discovery baselines under comparable feedback and inference budgets, and also evaluating evolved harnesses on held-out tasks to assess whether the discovered improvements gen...
Yike Wang, Huaisheng Zhu, Zhengyu Hu et al.· arXiv.org· 18 citations
FLIP (FLipped Inference for Prompt reconstruction), a reference-free and rubric-free reward modeling approach that reformulates reward modeling through backward inference that enables reliable reward modeling in downscaled regimes where judgment methods fail, is proposed.