HALO-WA is proposed, a hybrid-attention latent-guided online reinforcement learning (RL) framework for WA models, which leverages latent features and action priors from the WA generation process through a lightweight actor-critic adapter to enable fast online adaptation to real deployment errors.
An-Gen Ye, Wei-Jie Ke, Xiao-Feng Wang et al.· arXiv.org· 1 citation
Neural world models coupled with model predictive control (MPC) replan at every environment step to bound accumulated prediction error, but this incurs substantial computational overhead. Reusing a cached plan reduces this overhead, yet its effectiveness depends on how prediction mismatch propagates through the local d...
Yutian Cheng, Xiaojian Ma, Xianhao Wang et al.· 0 citations
A causal-transformer warm-start for the computationally dominant attitude-manipulator stage of a two-stage sequential convex programming (SCP) framework is developed and demonstrated that learned warm-starts can reduce onboard optimization cost while steering SCP toward basins containing feasible, low-cost solutions.
Yuji Takubo, M. Adang, Mac Schwager et al.· arXiv.org· 2 citations
This work introduces AVP (Action with Visual Primitives), an end-to-end architecture that implements this visual-primitive-centric interface: the VLM infers the next-stage target and emits visual-primitive tokens that condition a flow-matching action expert, with supervision derived from end-effector kinematics.
An energy-aware reinforcement learning framework that jointly commands wheels, tilt servos and propellers through a single continuous policy, without prescribing locomotion modes is presented.
SAIL is a framework that reframes robot imitation as an iterative refinement problem capable of scaling with test-time compute, and utilizes Monte Carlo Tree Search, where each node is a complete trajectory and edges correspond to trajectory refinements.
Full-stack human-robot collaboration (HRC) can become brittle when replacing a planner, partner model, coordination policy, or controller changes the physical meaning of cross-layer signals. We introduce C2C, an object-centric cognition-to-control architecture that preserves these meanings through physical contracts. R...
Hao Zhang, Yisen Li, Ruize Geng et al.· 0 citations
Across five real-robot settings with all policies running on the same Jetson AGX Thor, HybridFlow improves normalized task performance over 16-step Diffusion Policy by 13-68 points with approximately eightfold lower action-generation latency.
Zhen Dong, Fu-Lin Chen, Jin-Na Fu et al.· 0 citations
While heterogeneous teams have typically been designed for well-specified missions with known semantics, generative intelligence, i.e., large language models (LLMs) and vision language models (VLMs), opens the possibility of teams that infer mission-relevant semantics and subtasks given high-level natural language spec...
Zachary Ravichandran, Fernando Cladera, Ankit Prabhu et al.· 0 citations
NavSecure sets a new standard for developing more robust and trustworthy autonomous driving systems, capable of handling the inherent dynamics and uncertainties of real-world environments, capable of handling the inherent dynamics and uncertainties of real-world environments.
Hong Ding, Ziming Wang, Yi Ding et al.· arXiv.org· 0 citations
RankQ, an offline-to-online Q-learning objective that augments temporal-difference learning with a self-supervised multi-term ranking loss to enforce structured action ordering is proposed, which shapes the Q-function such that action gradients are directed toward higher-quality behaviors.
A visuo-tactile WAM that encodes each fingertip independently, aggregates the resulting features through a finger- and pose-aware tactile compressor, and injects the tactile latent into a video diffusion world model for joint visuo-tactile world modeling is presented.
Hao-Ran Yuan, Ze-Kai Wang, Bo-Ning Shao et al.· 1 citation
A new system identifies and corrects unreliable object descriptions stored in a robot’s memory, improving the robot’s ability to interact with its environment.
Robots are getting smarter, but how can their hardware match that growth? New Microsoft Research findings show that moving AI inference beyond the robot can improve task success, boost efficiency, and support more advanced physical AI workloads. The post Offloaded inference for real-world physical AI robotics appeared first on Microsoft Research.
