Skip to content
Preprint

IADD-TR: Intervention-Aware Dynamics Decoupling with Targeted Regularization for Model-Based Reinforcement Learning

Aug 2026 · 0 citations
Computer Science

TL;DR

IADD-TR is proposed, a unified framework combining Intervention-Aware Dynamics Decoupling (IADD) and Targeted Regularization (TR), which achieves competitive returns with improved sample efficiency on five MuJoCo tasks.

Abstract

Model-based reinforcement learning (MBRL), which learns environment dynamics to generate synthetic experience, is a promising approach to sample-efficient decision making. Numerous methods have been developed to improve dynamics prediction and policy optimization for MBRL through uncertainty estimation, model regularization, and conservative value learning. However, these methods typically treat the transition model and critic as monolithic predictors, overlooking the policy-induced data bias. Consequently, action can become entangled with environmental evolution, while uneven action coverage may distort the counterfactual value estimates used for policy improvement. To address this, we propose IADD-TR, a unified framework combining Intervention-Aware Dynamics Decoupling (IADD) and Targeted Regularization (TR). IADD factorizes transitions into an action-intervention stage and an action-free natural evolution stage, using a zero-action anchor to resolve the non-uniqueness of this two-stage factorization for robust generalization. Its latent and state-aligned components are identifiable up to an invertible within-block transformation and pointwise, respectively. For policy learning, we derive TR from the efficient influence function of a replay-state policy-gradient functional. TR augments the critic with an action-density-scaled residual correction and optimizes a targeted loss, yielding doubly robust policy-gradient estimation when either the critic or the replay action density is consistently specified. Extensive experiments on five MuJoCo tasks show that IADD-TR achieves competitive returns with improved sample efficiency.

View source

Similar papers

MA-JEPA: Joint-Embedding World Models for Multi-Agent Reinforcement Learning

World models improve sample efficiency by training policies on imagined trajectories, but their usefulness depends on learning representations that capture the information needed for future control. We study whether self-supervised joint-embedding prediction (JEPA) can provide this learning signal for multi-agent reinf...

Brandon Kaplowitz, Osaze James Obahor, C. S. D. Witt · 0 citations
Preprint Aug 2026

Dreamer-SAC: Off-Policy Learning in Latent World Models for Sample-Efficient Autonomous Driving

The Dreamer-SAC framework, which integrates a recurrent state-space world model with an off-policy soft actor-critic algorithm trained directly in latent space, uses a combination of real interactions and short-horizon generated trajectories with n-step target estimation and multi-objective supervision.

Jia-Zhuo Li, Lin-Jiang Cao, Qi Liu et al. · 0 citations

From Weak Data to Strong Policy: Q-Targets Enable Provable In-Context Reinforcement Learning

Q-Target Pretrained Transformers is proposed, which keeps the context-conditioned Transformer architecture but replaces behavior cloning with a Bellman-style Q-target objective, and learns to use rewards and transitions in the context to estimate action values, rather than simply imitating the behavior policy.

Yi-Chen Lin, Xu-Yuan Xiong, Xue Wang et al. · 0 citations
Preprint Aug 2026

Decoupling Policy Extraction for Offline Reinforcement Learning

This work revisits the conventional offline RL paradigm and proposes decoupling policy improvement from actor training, and trains the actor solely to model the behavior distribution and perform policy improvement at inference time by reranking multiple actor-generated proposals with a separately learned critic.

Xu-Yao Lin, Yixiang Shan, Jin-Ru Duan et al. · 2 citations
Preprint Sep 2026

Scouting the Dynamics Gap: Test-Time Policy Adaptation via Action-Outcome Feedback

SCOUT is a dynamics-aware meta-learning framework that enables manipulation policies to rapidly adapt by continuously revising their internal beliefs about environment dynamics via a shared belief latent space and significantly accelerates online adaptation across simulated manipulation benchmarks and achieves robust s...

Yi-Shu Li, Li-Yu Geng, Xin-Yi Mao et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.