Sep 2026· Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence· pp. 8208-8213· 0 citations· 41 references
TL;DR
An agenda for a shared MDP vocabulary, uncertainty-aware oversight, and scaling human-AI collaboration across volume, complexity, and expertise is opened.
Abstract
Learned agents that control traffic signals or call software tools are usually trained in simulators or fixed benchmarks, yet must act in worlds that differ. Many reliability concerns (robustness, safety, alignment, etc.) become one problem once agents are written as Markov decision processes (MDPs): a gap between the world an agent was built for and the world it acts in. The sim-to-real gap decomposes along state, observation, action, transition, and reward; multi-agent deployment further adds topology, population, and task gaps as neighbors learn and workloads scale. Using reinforcement learning (RL) for traffic control and large language model (LLM) agents for tool use and mobile GUIs as running examples, this paper traces how mismatches appear on each channel and how mitigations from one community (domain randomization, grounded action transformation, perturbation benchmarks) transfer to the other. Because the gap rarely closes, reliable deployment needs selective human oversight; uncertainty quantification can flag when to escalate, localize errors at the answer, reasoning, step, or agent level, and support sim-to-real transfer in RL. This paper opens an agenda for a shared MDP vocabulary, uncertainty-aware oversight, and scaling human-AI collaboration across volume, complexity, and expertise.
As agentic systems getting adopted rapidly in safety critical applications, it is vital to measure the confidence associated with the agentic actions. In comparison to the traditional machine learning systems, agentic workflows have complex failure modes with planning, tool invocation and dynamic environment interactions. In this paper, we investigate whether model's internal representations provide stronger signals of eventual task success in multi-turn agentic setups. We introduce two complementary methods: Latent Trajectory Dynamics (LTD), which summarizes changes in residual-stream representations across an an interaction trajectory, and the Action Representation Probe (ARP), which predicts success from representations formed at action decisions. Across three interactive benchmarks (Bash, SQL, Python) and three model families (Qwen14B, Qwen7B, DeepSeek6.7B), our methods consistently outperform surface level generation and sequence-based calibration baselines providing a zero-overhead reliability monitor that requires neither prompt alterations nor multi-sample rollouts.
Priyanka Mary Mammen, Emil Joswin, Srujananjali Medicherla· 0 citations
SIR is presented, a black box IPI attack that composes stealthy injections from a small library of reusable principles stated in plain language and wraps composition in an iterative feedback loop that diagnoses the victim's failed trajectories and distills the bypasses into new, named strategies that are reapplied across tasks.
Chen Xiong, Zhi-Yuan He, Pin-Yu Chen et al.· 0 citations
An action-aware reinforcement learning method that combines a per-turn reward sequence rewarding both the discovery and commitment of gold code regions with an action-level advantage estimation scheme that isolates each action's credit by grouping turns sharing the same exploration context is proposed.
Doyeon Kim, Suyoung Bae, Yumin Lee et al.· 0 citations
This work introduces AgentMercury, a scalable framework for synthesizing executable environments from high-level business scenarios that improves substantially on both enterprise workflows and out-of-domain benchmarks spanning reasoning, coding, scientific computing, and tool use.
Reinforcement learning for coding agents increasingly relies on long-running agent harnesses to manage tool integration, repository contexts, and execution feedback. However, the native execution environments of these harnesses are inherently misaligned with policy-gradient training: environmental crashes and reward hacking corrupt outcome signals, while train-inference discrepancies decouple rollout behavior from policy updates. To address this, we present LEGO-RL, a framework that bridges native coding-agent harnesses with scalable policy-gradient optimization without modifying their internal control flow. LEGO-RL is built upon three pillars: (1) faithful optimization via in-process LLM proxying that captures raw generation streams for token-level alignment and robust trainer-side log-probability recomputation, even under harness-side compaction or re-serialization; (2) reliable execution via scalable sandbox orchestration featuring image caching and stage-wise defenses to mitigate reward hacking; and (3) observable training through an integrated plugin that automates validation and monitoring, paired with a Live UI for granular trajectory diagnostics. We evaluate LEGO-RL by training the sparse MoE model Qwen3.5-35B-A3B with GSPO across three native coding-agent harnesses. LEGO-RL improves Qwen3.5-35B-A3B across OpenHands SDK (64.0% to 70.4%), Claude Code (62.4% to 68.2%), and OpenCode (57.2% to 66.6%) on SWE-bench Verified, while maintaining a rollout-training probability correlation above 0.99.
Yi-Ming Du, Yuxin Jiang, Tao Yuan et al.· 1 citation
This work proposes the Evolutionary Markov Hypergraph Attack (EMHA), a black-box policy that performs feedback-driven environment evolution by coordinating authorized state transitions without requiring parameter updates, and establishes OpenART as a scalable foundation for studying agent safety in complex, evolving environments.
Yunhao Chen, Xin Wang, Yixu Wang et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
