MaskVLA, a masking-based fine-tuning strategy that randomly masking a small portion of the main camera's visual information leads to the emergence of robust policies, thereby enhancing the model's capability to tackle complex manipulation tasks and improving its generalization performance.
Yuxuan Jiang, Jia-Ying Huang, Ge Wang et al.· 0 citations
RoboTalk is introduced, a synthetic data-generation pipeline and dataset of 7,950 multimodal trajectories spanning 53 mobile-manipulation kitchen tasks for training small VLMs to communicate and coordinate and fine-tuning open-source models on this dataset can reach 77% success on novel held-out tasks.
Dorian Benhamou Goldfajn, Mason Nakamura, Saaduddin Mahmud et al.· 0 citations
Overall, AR did not significantly improve cognitive load or usability compared with conventional instruction, but exploratory analyses revealed a compensatory effect: spatial ability predicted higher usability and lower extraneous cognitive load in the control group, but not in the AR condition, suggesting AR mitigated...
Nicolas Leins, Muriel Fischer, Malte Teichmann et al.· 1 citation
The Extended Reality Universal Planning Toolkit (ERUPT), an extended reality (XR) system for interactive motion planning, is presented to introduce the open-source system to others who can use it as a base to develop immersive robot interaction applications.
Isaac Ngui, C. McBeth, A. Santos et al.· arXiv.org· 0 citations
Reach audiences
Advertise in front of researchers, engineers, and readers.
Body-cue recognition can support assistive robots, but benchmark accuracy does not guarantee reliable behavior under a robot-camera viewpoint. We present Nuni, a bedside robot prototype that treats a detected distress cue as a reason to ask rather than a reason to alert. We compare two X3D-UGT RGB appearance classifier...
Listener Differences in Human-Robot Interaction (LD-HRI), a game, dataset, and benchmark that evaluates speakers through human listener performance, is introduced, providing a foundation for designing more robust communication in human-robot and human-agent interaction.
Promise Ekpo, T. Vijay, Dhruv Mandalik et al.· 0 citations
This article proposes a user-side framework in which long-term embodied AI is represented through an agent-specific identity representation organized by at least three open identity-content domains: embodied-perceptual, psychological-behavioral, and relational-autobiographical.
Spatial intelligence, encompassing 3D perception and reasoning, is the essential next frontier of AI. Scaling current 3D vision-language models (VLMs) that rely on dense Transformers for spatial tasks incurs prohibitive computational costs. In this paper, we introduce 3D-MoE, a 3D VLM leveraging an efficient mixture-of...
GraRe is proposed, which estimates grasp quality from candidate attributes, shell-stratified local geometry, and object context, and which shows that improving candidate ranking provides a practical way to enhance frozen 6-DoF grasp detectors.
Ji-Bao Yuan, Yuhui Zhao, Yinzhen Lv et al.· 0 citations
I describe my solution to the LeHome Challenge 2026, an ICRA 2026 competition on bimanual garment folding. The system placed 1st of 62 teams in the online (simulation) round and 2nd in the real-world final. It improves a vision-language-action (VLA) policy with a reinforcement-learning loop. The policy is its own value...
InSight is a framework for self-guided skill acquisition that uses a VLM to identify primitives missing from a VLA's repertoire, grounds the VLM's proposals through robot execution, and distills new primitives from successful rollouts into the VLA.
M. Wang, Lars W. Osterberg, Stephen Tian et al.· arXiv.org· 3 citations· ⚡1
The results demonstrate a common deployment path across VLA architectures and hardware, with numerical validation and control settings guiding deployment alongside inference efficiency.
Khanh-Binh Nguyen, Hung T. Ho, C. T. Nguyen et al.· arXiv.org· 2 citations
A new system identifies and corrects unreliable object descriptions stored in a robot’s memory, improving the robot’s ability to interact with its environment.
Robots are getting smarter, but how can their hardware match that growth? New Microsoft Research findings show that moving AI inference beyond the robot can improve task success, boost efficiency, and support more advanced physical AI workloads. The post Offloaded inference for real-world physical AI robotics appeared first on Microsoft Research.