This paper proposes a two-stage black-box framework to uncover and mitigate Positional Blind Spots, and evaluates its framework on five state-of-the-art VLA policies across two benchmarks, and finds that PBS are pervasive and spatially concentrated in all of them.
Abstract
Recent Vision-Language-Action (VLA) models achieve promising performance in robotic manipulation, typically measured by success rates aggregated over predefined object configurations, an evaluation that implicitly assumes spatially uniform competence across the workspace. However, this assumption does not hold: even with the instruction and every other scene factor held fixed, merely relocating a task-irrelevant distractor can sharply raise the failure probability within localized, spatially coherent regions, which we term Positional Blind Spots (PBS). In this paper, we propose a two-stage black-box framework to uncover and mitigate PBS. During the uncovering stage, we grid the workspace and apply a one-sided log-likelihood-ratio test to localize PBS cells with significantly elevated risk. During the mitigation stage, we fine-tune the policy via LoRA on demonstrations collected from these PBS regions, improving competence there while largely preserving performance across the rest of the workspace. We evaluate our framework on five state-of-the-art VLA policies across two benchmarks, and find that PBS are pervasive and spatially concentrated in all of them, with failure rates up to 0.58. Our search strategy achieves an average F1-score of 0.678, outperforming random search and adaptive sampling baselines by 0.268 and 0.178, respectively. Guided by the discovered regions, targeted fine-tuning reduces the overall failure rate by 40.00%--85.19%.
AtVLA, a framework that inserts learnable register tokens into the visual encoder and improves the average LIBERO success rate, is introduced, a framework that inserts learnable register tokens into the visual encoder and improves the average LIBERO success rate.
Jin Cui, Yanbin Hu, Xinyue Long et al.· 0 citations
Spatial intelligence requires more than describing object locations. Under incomplete observation, models must identify and acquire missing evidence, interpret it in a common spatial frame, and act on it. We introduce VA-Bench to evaluate the complete observe-reason-act-revise loop. General-purpose MLLMs learn procedural context from RGB-only demonstrations, actively select camera viewpoints, issue metric Cartesian commands, and revise them from execution feedback. Models receive no privileged object poses, oracle trajectories, or learned action heads. A fixed model-agnostic controller executes only model-specified targets. VA-Bench contains 14 base task families (11 single-arm and three dual-arm), seven held-out geometry/layout variants, and a long-horizon five-object composition track. We evaluate 12 primary model conditions in three independent runs over the same 20 physically verified seeds per base task, reporting terminal success, nine trajectory-level behavioral diagnostics, and subtask progress. First, the best-performing model scores 100.0% on target localization and 78.9% on spatial relations in the annotated run. Its three-run macro-average task success is only 53.93+/-3.17%. Second, active camera control significantly improves task success over passive multi-view observation. In one matched comparison, success rises from 27.86% to 57.50%. Third, held-out geometric transfer can reduce task success by over 30 percentage points. No model completes a strict long-horizon episode, despite substantial partial progress. VA-Bench thus tests whether general-purpose MLLMs can turn visual demonstrations and actively acquired evidence into successful embodied action.
Zhong-Bo Zhang, Jia-Yi Jin, Yi-Fan Wang et al.· 0 citations
Vision–Language–Action (VLA) policies remain brittle under modest distribution shift. On LIBERO-Plus, contemporary models that solve clean tasks at high rates can fall below 30% success when the camera’s viewpoint or the robot’s initial pose is perturbed. Most training-free test-time remedies address this problem through the image stream, for example, by augmenting, purifying, or selecting visual observations. In our controlled evaluation, this family of methods improves mean success by only about three points and leaves the robot-initial-state failure largely unresolved. This paper studies the failure at the level of input streams. A VLA receives visual tokens, a proprioceptive state token, and language tokens; different perturbations can move different streams away from their training manifold. In particular, the robot-initial-state perturbation directly shifts the proprioceptive token; therefore, image-space interventions have limited leverage. We introduce Gated Per-Stream Manifold Restoration (G-PSMR), a training-free wrapper for a frozen policy. For each stream, a lightweight gate detects off-manifold inputs and applies a stream-specific restoration before the policy forward pass. We instantiate the framework with entropy-gated visual consensus and gated relative-orientation debiasing, which preserves the within-episode orientation trajectory. In the original 280-episode paired evaluation, the joint method improves total success by +5.3 points compared with a +3.2 image-only gain and raises the most fragile factor from 20% to 38%. On 1120 previously unevaluated, manifest-disjoint task instances, the state restoration improves robot-initial-state success from 23.8% to 28.7%; a separate prospectively specified confirmation on 600 new gate-active instances yields 26.3%→32.2% (+5.8 points; 95% CI [+3.2,+8.5]; p<0.001). Together, the original cross-stream results and two independent state-stream evaluations support the central principle of matching the restoration to the input stream carrying the shift.
The real-robot benchmark demonstrates that StellaVLA can use both human/robot demos and human-to-robot (XR) demos as in-context structured demonstration to help VLA model adapt to OOD tasks.
Siyu Xu, Yunke Wang, Zijian Wang et al.· 0 citations
The Modality Masking Mechanism (M3), an embarrassingly simple, training-only strategy that requires no architectural changes or large-scale robot pretraining, is introduced to improve robustness of query-based VLA policies for bimanual manipulation.
Dong-Zhou Cheng, Ziang Li, Yixiao Zhou et al.· 0 citations
Vision-language models (VLMs) are increasingly used as zero-shot controllers, but successful trajectories do not necessarily show that decisions are grounded in visual input: simulator dynamics and conservative action priors can produce favourable scores without meaningful perception. We investigate this with an input-ablation battery: blind-image controls, repeated identical inputs, lane-axis reflection, non-visual baselines, and pipeline-integrity checks. Across nine direct-action models, six structured local VLMs, and an exploratory VLM-MPC hierarchy, we analyse 32,874 scored calls over two embodiments and three simulators. The direct-control results are largely negative: a constant-SLOW policy outperforms a scripted geometric controller, several models are image-invariant or nearly constant, and models that recognize longitudinal hazards still fail to transform LEFT and RIGHT under reflection. No local VLM meets the joint longitudinal and lateral grounding criteria. However, an image-only deterministic positive control estimates the lead gap with 0.090 m MAE and exact mirror equivariance, confirming the stimuli carry sufficient visual information; the failures are modular, not universal. A post-hoc, leakage-controlled symmetry-consensus guardian selects two models from 16 calibration frames and freezes a 2-of-4 hazard vote across original and reflected views. On 272 held-out frames it reaches 0.954 balanced accuracy (episode-cluster bootstrap 95% CI [0.895,0.990]); nested leave-one-episode-out recovers the same pair and threshold in all 12 folds. Abstaining on ties raises committed balanced accuracy to 0.973 at 0.824 coverage. With deterministic perception retaining lateral authority, offline modular replay achieves 0.934 action agreement and exact mirror equivariance. These results support current VLMs as bounded, selective hazard assistants, not monolithic zero-shot controllers.
J. de Curtò, Dayani Plasencia, Diego Sánchez et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
