The project addresses whether training regime changes the reliability of this stopping signal, as opposed to merely the accuracy of the policy, and evaluates self-consistency-based adaptive sampling across three training regimes of the same base model on the Countdown arithmetic task.
Quadrant-weighted Sampling for Length-aware Policy Optimization (QLPO), a simple resampling-based variant of GRPO that introduces implicit length control without modifying the reward function, suggests that structured resampling provides an effective and robust approach to efficient reasoning.
Desc descriptive evidence is provided that long-horizon multi-tool post-training can change ways of working that transfer beyond its training domain, and both software-engineering benchmarks improve despite the training collection containing no software-engineering tasks.
Sushant Mehta, Logan Ritchie, Liudas Panavas et al.· 0 citations
Online Self-Weighted Fine-Tuning is proposed, a simple method that augments SFT with online, trajectory-level weighting and offers a favorable compute-performance trade-off as a practical approach for fine-tuning small-to-medium LLMs on binary-verifiable reasoning tasks with only 2 online rollouts.
Hai-Quan Wen, Yi-Wei He, Bei Peng et al.· 0 citations
Best Practice Critic Optimization (BPCO) is developed, a recipe that combines DPPO, value predictions bounded to the reward range, Monte Carlo value targets, unnormalized policy advantages, and length-adaptive generalized advantage estimation and shows that a carefully designed critic provides a reliable alternative to...
DIEM is proposed, a principled and fully automated framework that makes data utilization adaptive throughout RFT and consistently outperforms strong static and dynamic baselines.
Haoru Tan, Sitong Wu, Yan-Feng Chen et al.· 0 citations
This work proposes ARMOR (Anchor Rollout and Mixed Optimization for RL), a framework that shifts the paradigm from passive penalty to active sample stabilization, enabling sustained performance improvements over extended training horizons.
Kexin Huang, Junkang Wu, Jinda Lu et al.· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
