Long-horizon research agents must decide both how to investigate and what to investigate next as evidence accumulates. This is hard to learn because such decisions are sparse in long execution traces, and their consequences may emerge several investigations later. We introduce Meta-reasoning for Iterative Research Agen...
Ankur Samanta, Yonathan Efroni, Paul Sajda et al.· 0 citations
Practical LLM agents often operate over multi-turn conversations where success is determined only after the full interaction ends. Most multi-turn RL methods train via on-policy rollouts, but unlike in single-turn RLHF, the policy cannot produce a trajectory alone, since an external environment must respond after each...
Daniel Jiang, Ankur Samanta, Yukai Yang et al.· 2 citations
Off-Context GRPO (OC-GRPO), a minimally modified variant of GRPO that uses guided rollouts but applies an importance-corrected objective to steer the update back toward the original unguided objective, avoiding the mismatch that destabilizes uncorrected guided training.
Priyank Agrawal, Ankur Samanta, S. Ghasemlou et al.· arXiv.org· 1 citation
This work develops a black-box method for learning a single context-independent logit-bias vector, added at every decoding step, without modifying model weights or requiring gradients, and suggests that learned logit bias is a lightweight mechanism for adapting language models under minimal access requirements.