Generative Artificial Intelligence (GenAI) constitutes a transformative technological wave that reconfigures industries through its unparalleled capabilities for content creation, reasoning, planning, and multimodal understanding. This revolutionary force offers the most promising path yet toward solving one of enginee...
Yuping Wang, Shuo Xing, Cui Can et al.· 0 citations
Ledger is presented, a persistent 3D object memory that combines object locations, their histories, and contextual descriptions, and clusters each object's observations by resting locations and records a move only after repeated evidence, reducing the effect of localization noise.
Shravan Chaudhari, William Paul, Suchi Saria et al.· 0 citations
Long-WAM is presented, a model-system framework for scaling the context of causal world-action models under real-time control constraints, and finds that access to history is not the same as using it: longer histories pay off far more when the video foundation is pretrained autoregressively (AR).
Wei Huang, Bo-Han Zhang, Chen-Zhi Liu et al.· 1 citation
We present FoldBack, a self-correcting masked generative policy for long-horizon garment folding. Existing long-trajectory policies may continue after a missed or slipped grasp even when the garment has not reached the intended configuration. We structure FoldBack's recovery mechanisms around three inference-time decis...
Lipeng Zhuang, Shiyu Fan, Yingdong Ru et al.· 0 citations
Reach audiences
Advertise in front of researchers, engineers, and readers.
Flow policies capture rich and diverse action distributions, and fine-tuning them with off-policy RL to improve beyond the demonstrations has drawn growing interest. However, fine-tuning a flow policy against a learned value function is not trivial, because the policy generates its action over many flow steps. Adjoint...
Yong Dong, Minsung Yoon, Jaehyuk Kim et al.· 0 citations
Recent advances in multimodal foundation models have made them capable generalist physical agents for a range of manipulation tasks. However, successful operation in an unfamiliar environment may require an agent to seek task-relevant information through interaction when it is absent from the observations: it may need...
Ren-Hang Liu, N. Majumder, Tej Deep Pala et al.· 0 citations
Human-Robot Collaboration (HRC) can facilitate mass customisation in Industry 4.0, with Reinforcement Learning from Human Feedback (RLHF) representing a promising approach for developing safe AI-based robots. Practical challenges remain regarding safety during AI development, human feedback quality, and bidirectional h...
Alexandra Coroiu, Andrea Vogt, Viktor Werbilo et al.· 0 citations
Offline reinforcement learning enables reward-driven policy improvement from fixed datasets without requiring online exploration, making it particularly attractive in safety-critical domains. A central challenge, however, is distribution shift: policy optimization may favor actions that are weakly supported by the offl...
Mahmoud Selim, Cristina Cipriani, K. H. Johansson· 0 citations
This work introduces an architectural framework that enables robotic control by leveraging generative AI in a reactive, constrained loop enables robust fulfillment of complex intentions in dynamic and unknown environments.
Justus Flerlage, Thorsten Wittkopp, Alexander Acker et al.· Lecture notes in computer sc...· 0 citations
Real-world applications require 6D pose estimation to be accurate, fast, and scalable to unseen objects. This paper introduces WAPR, a zero-shot wide-angle pose refinement model that refines candidate poses with rotational deviations up to 90 degrees. With as few as 12 candidate poses per detected object instance, WAPR...
Yu-Lin Wang, Meng-Ting Hu, Hong-Li Li et al.· 0 citations
Large vision-language models (VLMs) provide powerful open-world perception and reasoning for autonomous driving, but their high computational cost and inference latency make continuous cloud-side use impractical. This motivates fast--slow collaboration, where efficient onboard modules handle real-time perception and co...
Jia-Yi Chen, Shuai Wang, Guang-Xu Zhu et al.· 0 citations
Vision-Language-Action (VLA) models have shown strong generalization in robotic manipulation by leveraging rich representations from pretrained vision-language models. However, their deployment in real-world environments remains limited by recurring unreliable behaviors. In this work, we study state hallucination, a re...
Jiho Lee, Jeongeun Park, Heayoun Choi et al.· 0 citations
A new system identifies and corrects unreliable object descriptions stored in a robot’s memory, improving the robot’s ability to interact with its environment.
Robots are getting smarter, but how can their hardware match that growth? New Microsoft Research findings show that moving AI inference beyond the robot can improve task success, boost efficiency, and support more advanced physical AI workloads. The post Offloaded inference for real-world physical AI robotics appeared first on Microsoft Research.