Skip to content

Safe Real-Time Policy Steering via Noise-Space Trajectory Optimization for One-Step Generative Policies

Sep 2026 · arXiv.org · Vol abs/2609.21220 · 1 citation · 30 references
Computer Science

TL;DR

Insight-time steering of one-step generative policies as trajectory optimization in the policy's input noise space is proposed, which improves task success and constraint satisfaction over best-of-N sampling and action projection, while comparing favorably with gradient-guided generation at lower runtime.

Abstract

Generative robot policies can represent diverse, multimodal behaviors, but adapting pretrained policies to deployment-time constraints such as collision avoidance and orientation maintenance remains challenging. Existing inference-time steering methods typically apply gradient guidance through iterative diffusion or flow processes, which can be computationally expensive for real-time control. We propose INSPO, which formulates inference-time steering of one-step generative policies as trajectory optimization in the policy's input noise space. By optimizing the input noise while evaluating constraints on the induced state trajectory, INSPO searches the policy-induced behavior space without directly modifying generated actions. The optimization includes a regularization term that encourages solutions to remain consistent with the policy's input distribution and is solved online using population-based particle optimization. We evaluate INSPO on state- and image-based task-specific policies and generalist vision-language-action policies across Push-T, Can pick-and-place, and LIBERO-Spatial. INSPO improves task success and constraint satisfaction over best-of-N sampling and action projection, while comparing favorably with gradient-guided generation at lower runtime.

View source

Similar papers

Open access Dec 2026

LWAT: Learning Width-Aware Trajectory-Level Actions for Real-Time Robot Motion Planning

Generating smooth and safe trajectories from onboard sensor data remains a fundamental challenge for mobile robots operating in cluttered and dynamic environments. Conventional deep reinforcement learning planners commonly output instantaneous control actions, which often leads to short-sighted decisions, fragmented mo...

Wan Guo, Wen-Fang Xie, P. Arab et al. · 0 citations
Preprint Oct 2026

RFPO: Rectified Flow Policy Optimization for Embodied Control

Flow-based policies provide an expressive framework for continuous robot control, but their iterative ODE integration incurs substantial inference cost. Naively reducing the integration budget can severely degrade control, since policies optimized under full-step execution are not explicitly constrained to remain relia...

Ting Huang, Li-Rui Pan, Hao-Yu Wang et al. · 2 citations
Preprint Nov 2026

Fast Inference-Time Policy Refinement via Action-Affine World Models

Imitation policies can fail through compounding errors, while existing refinement methods often require additional on-policy learning or expensive online planning. Our key insight is to model the response to small corrections around a policy proposal in an affine form that enables fast optimization. We refine a frozen...

Run Wang, Yi-Lin Mo, Liang Wu · 0 citations
Preprint Nov 2026

APG-Planner: Learning Vision-Based High-Speed Trajectory Planning via Analytic Policy Gradient

Vision-based high-speed navigation requires a trajectory planner that can reactively generate smooth, collision-free trajectories from onboard visual observations. Deep reinforcement learning offers a promising approach to vision-based trajectory planning, as it enables the optimization of sequential decision-making po...

Jin-Woo Lee, Geunsik Bae, Changseung Kim et al. · 0 citations
Preprint Nov 2026

GUARD: Anomaly-Informed Q-Steering for Generative Robot Policies in Dexterous Manipulation

Generative robot policies based on diffusion and flow matching can model complex, multimodal behaviors, but remain susceptible to covariate shift during closed-loop execution. Existing approaches mitigate such failures through additional demonstrations, policy fine-tuning, or test-time guidance, but often require human...

Aryaman Gupta, Xia Wu, Rana Soltani Zarrin et al. · 0 citations
#machine learning Preprint Oct 2026

DR-APO: Decoupled-Representation Advantage-Guided Policy Optimization from Continuous Control to Embodied Navigation

DR-APO (Decoupled-Representation Advantage-Guided Policy Optimization) is introduced, which decouples policy improvement from policy representation through advantage-guided target improvement and representation-compatible direct action fitting, providing a shared optimization principle for Gaussian, diffusion, and flow...

Run Zhang, Ying-Jie Niu, Chao Li · 1 citation

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.