Across diverse benchmarks and world-model families, BasinLens exposes reproducible and locally persistent failure modes that conventional evaluations fail to reveal, showing that average-case benchmarks can mask important vulnerabilities in world-model-driven control.
Abstract
World models predict action-conditioned futures and serve as critical internal simulators for downstream planning and control. However, catastrophic prediction failures of world models could dangerously propagate through the control pipeline, as subsequent agent or model training and decision-making depend heavily on the continuous environment evolution forecasted by these world models. Existing evaluations overlook this systemic risk: by aggregating average errors over benign generations from general queries, they fail to stress-test the model against catastrophic collapses under rare or unobserved condition-action combinations. To bridge this gap, we formalize the natural-input failure discovery problem: under a finite query budget, finding environment-valid conditions and action prefixes that induce severe prediction risk, verifying whether these failures reproduce on fresh seeds, and testing their persistence under nearby valid edits. Discovering such critical failures is computationally challenging, as valid condition-action combinations explode exponentially, rendering exhaustive search or standard sampling infeasible given the high cost of noisy rollouts. To tackle this, we propose BasinLens, which exploits the underlying structure of valid inputs, where each coordinate possesses environment-defined semantic types and admissible domains, by pairing uncertainty-guided global search with typed local replacements. Across diverse benchmarks and world-model families, BasinLens exposes reproducible and locally persistent failure modes that conventional evaluations fail to reveal, showing that average-case benchmarks can mask important vulnerabilities in world-model-driven control.
A training-free framework that reformulates failure recovery as test-time scaling over causal histories, which decomposes recovery into three coupled decisions: when to revise the causal history, where to recover a reliable history prefix, and which history configuration best supports subsequent execution.
Lin Li, Long Chen, Kwunhang Wong et al.· arXiv.org· 0 citations
Speculative Uncertainty (SU), a method that recovers a predictive failure signal for a black-box agent from its output tokens alone, with no access to logits, weights, activations, or repeated sampling, is presented.
Mid-Harness is introduced, which samples and verifies candidate actions before forwarding one for execution, while keeping the generator and harness unchanged, and identifies action scaling as a promising target for test-time compute scaling in terminal agents.
This study introduces SymTrace, a controlled evaluation framework that records the MAS execution trajectory and establishes intervention anchors and explores the effectiveness of MAS repair methods, revealing that existing unguided rerun methods are highly unreliable.
Zhong-Wen Luan, Xiaoyan Zhang, Ming Hu et al.· 2 citations
Preventing transient control loss in risk-critical human-machine systems requires dynamic intervention strategies that account for unobservable user states. Traditional control frameworks relying on fully observable Markov decision processes are inadequate for this task, as they inherently suffer from perceptual aliasi...
Oleksandr Chaban, V. Hladun· Automation, Control, and Inf...· 0 citations
Pretrained world models, learned simulators that encode an observation into a latent state and predict how it evolves under actions, are beginning to be reused as off-the-shelf dynamics backbones for control, like pretrained encoders and language models are reused today. We show that this reuse opens a supply-chain bac...
Roberto Riaño, Gorka Abad, S. Picek et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.