Sampling-Guided Policy Search (SGPS), which couples recurring action-target refinement by sampling-based model-predictive control with first-order policy optimization with first-order policy optimization, and shows that refinement improves policy learning beyond initialization and tracking alone.
Abstract
Learning visual policies for locomotion and manipulation requires coordinating contact with the environment and can incur substantial computation and GPU memory costs. First-order policy gradients (FoPG) reduce training cost through differentiable simulation, but local optimization can converge to unintended contact patterns. To address this shortfall, we propose Sampling-Guided Policy Search (SGPS), which couples recurring action-target refinement by sampling-based model-predictive control with first-order policy optimization. Behavior cloning initializes the policy from sampled actions; training then alternates sampling-based refinement with short-horizon FoPG updates under perturbed initial states and randomized dynamics. For visual policy training, we use a decoupled FoPG formulation that excludes rendering from the computation graph, enabling direct learning from depth observations without a state-policy teacher. On a single GPU, SGPS learns policies for locomotion, obstacle traversal, crate pushing, and bimanual carrying on simulated Unitree Go2 and G1 robots. Our experiments further show that refinement improves policy learning beyond initialization and tracking alone. For hardware deployment, the distilled policy transfers zero-shot to a real Go2 and uses onboard depth to autonomously trot, crawl, clear hurdles, and switch between these behaviors.
This work introduces Visual and Motor Policies from Sampling-Based Planning (VAMPS), a framework that uses Model Predictive Path Integral (MPPI) control to train reusable policies without human demonstrations, and demonstrates visuomotor pick-and-place and force-aware whiteboard erasing.
Mohamed Yassine Kabouri, Pietro Noah Crestaz, Q. Nguyen et al.· 0 citations
Traversing narrow gaps is challenging for autonomous quadrotors, especially when control commands come directly from high-dimensional visual observations. Existing end-to-end methods often rely on behavior cloning or full-rollout backpropagation through time (BPTT) via differentiable simulation, which can limit policy...
Insight-time steering of one-step generative policies as trajectory optimization in the policy's input noise space is proposed, which improves task success and constraint satisfaction over best-of-N sampling and action projection, while comparing favorably with gradient-guided generation at lower runtime.
Qing-Yi Chen, Joseph Ruan, Zachary Kingston· arXiv.org· 1 citation
Copper-Policy is introduced, which learns a compact World representation with the policy rather than relying on a predefined target space and predicts future observation embeddings conditioned on task intention without reconstructing pixels through temporal joint-embedding prediction.
Ze Feng, Yi-Xu Feng, Ling-Yu Xiao et al.· arXiv.org· 1 citation
Group Relative Return Policy Optimization (GR2PO), a critic-free reinforcement learning framework for continuous robot control, is proposed and significantly outperforms critic-free baselines that use immediate rewards and performs competitively against state-of-the-art actor-critic methods.
GEM-MPC is proposed, an MPPI-based reinforcement learning method that improves the interaction between planning and learning and selectively learns from stored planning distributions only when they provide a better target than the current prior, reducing the impact of stale planning data without requiring full reanalys...
Álvaro Serra-Gómez, T. Moerland· arXiv.org· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduNov 9, 2026
Machine-learning networks can learn without detailed instructions from a designer. Those that do may offer insight into how the brain reorganizes itself during learning.
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.