AdaStart: Uncertainty-Aware Adaptive Warm Starting for Diffusion-Based Robotic Control

Abstract

Diffusion policies have demonstrated excellent performance in robotic control tasks, yet their reliance on 50 to 100 denoising steps impedes real-time deployment. Existing acceleration methods based on sampler improvements lack responsiveness to perceptual quality and cannot adaptively adjust computation. Moreover, standard diffusion policies often overlook the reliability of visual observations, leading to degraded robustness under occlusion or domain shift. To address this, we propose AdaStart, an uncertainty-aware adaptive warm starting method, employing a lightweight visual multitask auxiliary network to predict robot state and inform action generation. By estimating visual uncertainty from state prediction error, AdaStart dynamically selects the starting timestep of the reverse diffusion process and constructs a warm-start noisy action via the analytical forward diffusion of the predicted action. With minimal parameter overhead, it accelerates inference under high-confidence conditions while safely reverting to the full diffusion trajectory when uncertain. Experiments on multiple simulated and real-world manipulation tasks show that AdaStart reduces inference steps by nearly 40% while maintaining or improving task success and robustness.