To solve problems in 3D space rather than merely describe images, multimodal large language models (MLLMs) need 3D spatiotemporal reasoning: reconstructing 3D geometry from occluded 2D views, anticipating physical dynamics, and executing precise, closed-loop actions. Prior studies test passive VQA on static images or s...
Chenglong Shi, Zekai Yuan, Yue Song et al.· 0 citations
Zero-shot vision-and-language navigation (VLN) requires an agent to use past observations without navigation-specific training. We study what information an 8B-scale navigation system should retain and how it should be organized for subsequent decisions. Our agent combines instruction progress, hierarchically compresse...
Yuhang Peng, Zhikai Wang, Ziyi Wang et al.· 0 citations
Robot imitation learning uses demonstrations from operators of varying skill, but selecting training data remains challenging. Existing methods score demonstrations using smoothness, diversity, or estimated influence on policy performance, but differences in selection and evaluation protocols make it difficult to deter...
We introduce EARS-Info, an information-aware extension of Event-Driven Ant Reallocation Search for multi-robot exploration on OpenStreetMap-derived grids. It combines marginal sensing gain, feasible relocation, return-aware routing, sensing reservations, and cached shortest paths. Across 168,000 episodes comparing 30 i...
Kabeh Mohsenzadegan, V. Tavakkoli, K. Kyamakya· 0 citations
Reach audiences
Advertise in front of researchers, engineers, and readers.
Humanoid controllers often require task-specific training despite sharing the need for balance and whole-body coordination. We present Metatron, a promptable whole-body execution layer combining a generative base policy with a programmable skill layer. The base policy receives partial goals, collectively called a promp...
Chen Tessler, Yifeng Jiang, Alec Otto et al.· 0 citations
Vision-Language-Action (VLA) models offer a promising paradigm for general-purpose household robotics. However, inference and data-collection strategies that have been deployed in smaller-scale settings do not necessarily generalize to large-scale, diverse task suites. To address this gap, we conduct a systematic, pair...
Ruben Hammele, Samira Huber, Mohammad Nazeri et al.· 0 citations
The code, weights, evaluators, execution traces and cost accounting accompany the paper, where it is observed that the extra updates decrease the costs of the selected latent world-model trajectories for the updater, while the tasks that used to be successes become failures.
Initial results suggest that scaling off-policy experience is a promising approach to efficient direct visual RL, and a single-stage off-policy RL framework that scales experience collection without proportionally increasing visual computation is introduced.
This paper introduces OrbitalGym, a simulation environment for studying when planning algorithms or system design determine the outcome of adversarial orbital encounters. In an orbital game, two non-cooperative teams of satellites maneuver during rendezvous and proximity operations to win a differential-reward objectiv...
Duncan Eddy, Mahdi Al-Husseini, Mykel J. Kochenderfer· 0 citations
Planning at run time, as in robots that re-plan while they act, needs a planner that reads the PDDL its users write (durative actions, numeric resources, time windows, continuous change), answers quickly, and delivers only validated plans. Specialised planners are strong on individual fragments of PDDL, but they differ...
Francisco Martín Rico, Esther Aguado González, Francisco Miguel Moreno Olivo et al.· 0 citations
Zero-real Agent Policy (ZAP), a recipe that teaches a small off-the-shelf vision-language model to control a robot via a simple tool harness, uses a pretrained 27B parameter model to bootstrap world understanding and reasoning, followed by supervised fine-tuning and GRPO on simple simulated tasks to teach it how to mov...
Benjamin Pikus, D. Saunders, Mircea Mironenco et al.· 0 citations
VLA-HWBench is introduced, a benchmark that measures 18 open VLA models on NVIDIA Jetson AGX Thor and AMD Ryzen AI Max+ PRO 395, with a seven-stage inference breakdown, and reveals three insights that suggest low-latency edge VLAs should use few action-generation iterations, inference software should extend kernel fusi...
Bo-Rui Xu, Hong-Shi Tan, Wen-Qi Jiang et al.· 1 citation
A new system identifies and corrects unreliable object descriptions stored in a robot’s memory, improving the robot’s ability to interact with its environment.
Robots are getting smarter, but how can their hardware match that growth? New Microsoft Research findings show that moving AI inference beyond the robot can improve task success, boost efficiency, and support more advanced physical AI workloads. The post Offloaded inference for real-world physical AI robotics appeared first on Microsoft Research.
