The benefit of incorporating VLA capability into authority allocation to mitigate over-helping and improve shared-control performance is demonstrated, and the proposed method achieved the highest task success rate.
Abstract
Shared control often allocates robot authority based on confidence in inferred human intent, assuming reliable autonomous execution. When this assumption fails, high intent confidence can cause over-helping. We present a capability-aware shared-control framework in which a vision-language model (VLM) infers human intent and provides semantic-intent confidence, while a vision-language-action (VLA) policy generates autonomous actions. VLA capability confidence is estimated online from the dispersion and local instability of stochastic action trajectories. We design a nonlinear arbitration policy that combines Bayesian-filtered semantic-intent confidence with VLA capability confidence through a sigmoid mapping to adapt robot authority. Our evaluation combined VLM/VLA confidence assessment with a study involving 12 participants performing pick-and-place and bidirectional stacking under in-distribution and out-of-distribution conditions. The proposed method achieved the highest task success rate (92%), compared with manual teleoperation (83%), intent-only arbitration (44%), and fixed equal-weight blending (10%). It also achieved higher control friendliness and lower authority-weighted disagreement than both shared-control baselines. These results demonstrate the benefit of incorporating VLA capability into authority allocation to mitigate over-helping and improve shared-control performance.
Results show that directly steering internal robot-policy representations can improve the performance of robot policies at inference time and show that directly steering internal robot-policy representations can improve the performance of robot policies at inference time.
Jiu-Zhou Lei, Chang Liu, Da-You Li et al.· 0 citations
Large language models (LLMs) are increasingly used as high-level planners in robot navigation, but their outputs may become unreliable when instructions are ambiguous, unsupported by the environment, or semantically inconsistent. This paper presents a Risk-Aware Semantic Grounding framework for trustworthy LLM-based ro...
Łukasz Sobczak, Nur Keleşoğlu, S. Nowak· 0 citations
Assistive agents in shared-control settings face a dual challenge: acting proactively to align with the human's intentions while remaining legible enough for the human to comprehend and fluently respond in real time. We present Dream2Assist, a model-based reinforcement learning framework for tightly-coupled shared deci...
Jonathan A. DeCastro, T. Balch, Emily Sumner et al.· 0 citations
An event-driven framework that monitors workspace state changes with an event monitor and, upon event completion, extracts stabilized pre/post snapshots that characterize the resulting state transition achieves performance comparable to variants given user instructions.
Feng-Kai Liu, Hao Su, Hao-Zhuang Chi et al.· 1 citation
A memory-based agentic framework in which a VLM task predictor reads a Procedural Assistance Memory alongside the current observation to decide when to assist and what task to perform, and a VLA executor performs the predicted task.
Dohyun Lee, Minho Park, Jeonghoon Park et al.· 2 citations
Axiom-0, a VLA policy that addresses both the quality of its representations and their use in action generation through hierarchical self-refinement, is presented, a VLA model that combines pretrained multimodal understanding with learned control on unfamiliar task compositions.
Ren-Bin Shu, Jing Yang, Xiang-Yuan You et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.