This work presents CAST (Critic with Alternating State-value Target), which uses planner-guided behavior to improve value learning while regularizing the value estimate with the current policy.
Abstract
Model-based reinforcement learning (MBRL) is a family of RL methods that learn a model of the environment and use it for action selection, making it well suited to robotics due to its sample efficiency. Combining learned models with online planning can further improve action selection, as the planner can exploit the model to find better actions than the learned policy alone. Recent methods combining learned policies with online planning typically learn the value of the policy rather than the stronger planner-guided behavior. We present CAST (Critic with Alternating State-value Target), which uses planner-guided behavior to improve value learning while regularizing the value estimate with the current policy. CAST replaces the action-value critic with a state-value critic, trained using a target that combines a real planner-guided transition and an imagined transition under the current policy. The resulting value function corresponds to an alternating process between planner-guided behavior and the current policy, allowing it to benefit from the stronger planner behavior while being regularised by the policy being learned. We evaluate CAST on the DeepMind Control and HumanoidBench Suites against several state-of-the-art methods, and demonstrate successful transfer to a physical Unitree Go2 quadruped performing a dynamic handstand.
GEM-MPC is proposed, an MPPI-based reinforcement learning method that improves the interaction between planning and learning and selectively learns from stored planning distributions only when they provide a better target than the current prior, reducing the impact of stale planning data without requiring full reanalys...
Álvaro Serra-Gómez, T. Moerland· arXiv.org· 0 citations
This work introduces PL-MPC (Planning-Learning MPC), which additionally modifies critic supervision and planner terminal-value estimation, and demonstrates zero-shot sim-to-real transfer on wrench-nut alignment with a 7-DoF KUKA IIWA14, obtaining higher observed success than TD-M(PC)^2.
With massively parallel simulation, on-policy Reinforcement Learning methods such as PPO have become standard in many domains. However, learning from scratch is sample-inefficient and fails to exploit the potential existence of a suboptimal expert, such as a heuristic, a model-based controller, or a policy trained on a...
D. Affinita, Ming-Jing Xu, Rudolf Reiter et al.· 0 citations
Vision-Language-Action (VLA) models have emerged as a powerful paradigm for embodied intelligence, but fine-tuning them with reinforcement learning (RL) remains constrained by the cost of real-world robot interaction. Model-based reinforcement learning (MBRL) reduces this cost by using a learned world model to generate...
Reinforcement Learning (RL) has emerged to be one of the most radical ideas in artificial intelligence since it enables the learner to learn the desired behavior through a response to the environment. The survey provides the overall picture of reinforcement learning, its development history beginning with the classical...
N. Ramshankar, D.Elangovan, S. S et al.· 2026 10th International Conf...· 0 citations
Experiments on six synthetic two-dimensional target distributions with diverse geometries demonstrate that RFPO captures complex multimodal structure without mode collapse, and across six continuous-control tasks, RFPO matches or outperforms a standard Gaussian-policy baseline on almost every task.
B. Park, Hyukjun Yang, Donghwan Lee· arXiv.org· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.