This work introduces WirelessOptBench, a benchmark for action assurance in wireless operations that turns wireless tasks into execution state decision episodes with controlled telemetry faults and action constraints, and develops WirelessOpsAgent, which grounds candidate actions in current evidence and repairs recoverable support failures before execution.
Abstract
Large language model (LLM) agents are emerging as planners for autonomous wireless network operations. Yet a task answer that is correct at proposal time can still be unsafe at execution time if supporting telemetry is stale or inconsistent. Existing benchmarks mainly evaluate task solving from fixed observations and leave support checking at execution time untested. We introduce WirelessOptBench, a benchmark for action assurance in wireless operations. It turns wireless tasks into execution state decision episodes with controlled telemetry faults and action constraints. We further develop WirelessOpsAgent, which grounds candidate actions in current evidence and repairs recoverable support failures before execution. Across three backbone evaluations with 600 episodes each, WirelessOpsAgent achieves up to 0.983 Exact Action Accuracy. On Claude Sonnet 4.6, the Unsafe APPLY Rate decreases from 82.2% to 10.3% relative to the safest baseline. We make WirelessOptBench available at https://anonymous.4open.science/r/wirelessopsbench-artifact-D969/.
NetConfArena is presented, an executable benchmark for evaluating LLM agents in closed-loop network configuration, and its findings suggest two future directions: using validated trajectories as supervision signals to improve foundation models, and designing harness mechanisms that make agent execution more reliable an...
Chang Liu, Xiao-Hui Xie, Xinyi Chen et al.· 1 citation
ReAct-based agents typically rely on a single LLM policy to propose actions, interact with the environment, and decide when a task is complete. This coupling makes action authorization and completion control difficult to enforce independently, allowing errors to propagate and unsupported completion claims to terminate...
Ajay Vohra, Tao Chen, Neeti Narayan et al.· 0 citations
This work shows that agent verifiers leveraging internal signals predict both harm and progress more reliably than a baseline using observable signals only, and aims to use these signals as safety feedback to an agent harness to abstain from risky actions and protect the target system.
T. Labarta, Frederik Pahde, Novak Boškov et al.· 0 citations
RedAgentBench is introduced, an executable framework for autonomous red-teaming and faithful measurement that shows that executable evaluation can improve safety measurement and identify actionable intervention points.
Zixing Chen, Xingyuan Liu, Jie Zhu et al.· 3 citations
An agenda for a shared MDP vocabulary, uncertainty-aware oversight, and scaling human-AI collaboration across volume, complexity, and expertise is opened.
Hua Wei· Proceedings of the Thirty-Fi...· 0 citations
An agent can complete a task while exhibiting undesirable behavior during execution. Developers need tests for the specific behaviors encountered in deployment, beyond fixed benchmark suites. We present TraceDance, an agent system that constructs targeted benchmarks from deployment traces for user-specified undesirable...
De-Hai Min, Dao-An Zhang, Yiming Zeng et al.· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.