Dec 2026· IEEE Robotics and Automation Letters· Vol 11, pp. 13997-14004· 0 citations· 31 references
Abstract
Embodied agents executing tasks from natural language instructions face the challenge of interpreting ambiguous commands that may lead to inconsistent planning. Conventional uncertainty estimation methods, reliant on token-level probabilities or semantic embeddings, struggle to capture task-level intent divergence, while large language model (LLM) based planning suffers from computational overhead. To address these problems, this paper presents Intent Entropy, an innovative uncertainty quantification method that models planning-level ambiguity through structured task chain clustering. By computing entropy over equivalence-clustered subtask sequences, Intent Entropy directly measures intent uncertainty rather than linguistic variations, exhibiting statistically significant negative correlation with task success, substantially outperforming conventional uncertainty measures. Building upon this metric, we introduce the Entropy-Gated Reflection Mechanism (ECHO), a hierarchical framework that employs lightweight models for initial planning and selectively triggers large-model reflection only under high uncertainty. The proposed methodology is evaluated on the ALFRED benchmark, demonstrating superior performance on both seen and unseen scenarios while significantly reducing computational costs compared to baseline methods. Furthermore, ECHO exhibits robust performance across varying instruction ambiguity levels, showcasing its effectiveness in uncertainty-aware adaptive planning.
This work proposes a Bayesian framework that treats clarification as an active learning problem over grounded Signal Temporal Logic task specifications and uses LLMs to initialize candidate formal specifications and translate informative contrasts into natural-language clarification questions, while Bayesian optimizati...
Hu-Ao Li, Carson Sobolewski, A. Saravanos et al.· arXiv.org· 0 citations
Large language models (LLMs) are increasingly used as high-level planners in robot navigation, but their outputs may become unreliable when instructions are ambiguous, unsupported by the environment, or semantically inconsistent. This paper presents a Risk-Aware Semantic Grounding framework for trustworthy LLM-based ro...
Łukasz Sobczak, Nur Keleşoğlu, S. Nowak· 0 citations
The benefit of incorporating VLA capability into authority allocation to mitigate over-helping and improve shared-control performance is demonstrated, and the proposed method achieved the highest task success rate.
Zhao-Da Du, Michael Bowman, Xiao-Li Zhang· 0 citations
Reinforcement Learning with Verifiable Rewards (RLVR) has improved the reasoning capabilities of Multimodal Large Language Models (MLLMs), yet existing frameworks rely on coarse, sequence-level reward signals that lack the fine-grained supervision over the visually-grounded steps within a multimodal reasoning chain. We...
Zhi-Han Zhang, Li-Zi Liao· Proceedings of the 34th ACM...· 0 citations
Reinforcement learning with verifiable rewards has substantially advanced multimodal reasoning, yet it remains fundamentally limited by ambiguous token-level credit assignment. While high-entropy token heuristics encourage possibility exploration, naively extending them to video reasoning tends to induce lengthy reason...
Yu-Dong Han, Yong Wang, Zai-Quan Yang et al.· 0 citations
Multimodal large language models (MLLMs) can act unsafely when individually benign images and texts are interpreted jointly. Beyond implicit reasoning risk, where reasoning drifts toward harm, we identify intent-level risk, where cross-modal ambiguity silently commits the model to a hazardous premise that path-level op...
Wei Cai, Jian Zhao, Yu-Chen Yuan et al.· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduNov 20, 2026
Senior Research Scientist Jim Glass discusses how his MIT-IBM Computing Research Lab collaboration connects perception and language, and addresses the shortcoming of current models.
MIT News · Artificial Intelligence· news.mit.eduNov 20, 2026
Enforcing physics-based constraints prevents AI models from generating unreliable predictions for complex nonlinear systems in weather, climate, and turbulent flows.
An MIT and Dana-Farber Institute-led team used large language models to examine nearly 4,000 studies and hundreds of code repositories, revealing gaps in the information needed to scrutinize and reuse published research.
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.