Effective Exploration via Intrinsic Motivation in Reinforcement Learning

Integrating Novelty and Surprise for Experience Prioritization and Exploration in Image-Based Reinforcement Learning

Novelty and Surprise Prioritized Experience Replay (NSPER) is introduced, which uses novelty to capture underrepresented states and surprise to expose gaps in the agent's understanding of the environment and is extended with NSPER+R, integrating these signals as intrinsic rewards to jointly improve replay quality and exploration.

On-policy Reinforcement Fine-tuning with Offline reward for Multi-step Embodied Planning

Embodied planning requires agents to make coherent multi-step decisions based on dynamic visual observations and verbal goals. While recent vision-language models (VLMs) excel at static perception tasks, they struggle in interactive environments. Reinforcement learning (RL) offers a natural way to address this limitation, yet online RL approaches suffer from costly interaction and sparse rewards in embodied settings. This paper introduces ORBIT , an O n-policy R einforcement fine-tuning (RFT) framework with offline rewards for Em B od I ed T ask Planning, that preserves the generalization benefits of RFT while addressing the challenges of costly interaction and sparse rewards, supported by solid theoretical guarantees. Our approach is evaluated on EmbodiedBench, a recent benchmark for interactive embodied tasks, covering both in-domain and out-of-domain scenarios. Experimental results show that ORBIT achieves SOTA performance on EB-ALFRED, outper-forming all closed-source and online-RL-based methods, while being substantially more effi-cient in training speed and computational cost, remaining robust to sub-optimal expert trajectories, and exhibiting strong generalization to unseen environments. We released all code and data at https://github.com/mail-taii/Reinforced-Reasoning-for-Embodied-Planning

Adaptive Reward Design in Reinforcement Learning: A Taxonomy and Survey

A Unified Framework for Dynamic Reward Shaping in Reinforcement Learning

Sparse, delayed, and weakly informative rewards remain central obstacles to efficient reinforcement learning. Reward shaping addresses these limitations by supplementing the task reward with an auxiliary signal that can accelerate learning while, in the classical setting, the original objective remains the evaluation criterion. Established theory guarantees safety for fixed shaping signals: potential-based reward shaping preserves optimal policies when the auxiliary term is the discounted difference of a time-invariant potential. In contemporary reinforcement learning systems, however, both the learner and the information available for guidance evolve during training: value estimates improve, novelty diminishes, feedback shifts, and predictive models are refined. Adaptive reward mechanisms occur across exploration, Bayesian inference, human-in-the-loop learning, automated reward design, and foundation-model-based approaches. This study introduces a unified analytical framework for comparing dynamic reward shaping and neighbouring adaptive reward mechanisms. The proposed framework distinguishes parametric revision from state-dependent variation, separates additive shaping from reward replacement and reward-adjacent guidance, and organises existing methods along temporal, informational, and theoretical dimensions. Using this framework, twelve method families are comparatively analysed. The framework further highlights the conditions under which optimality guarantees survive contemporary deep reinforcement learning pipelines, replay buffers, bootstrapped critics, and reward normalisation, while exposing the unresolved relationship between adaptation rate and learner stability.

Exploration and Exploitation: A Study on Sample Efficiency in Reinforcement Learning With Multifaceted Curiosity Rewards and Adaptive Experience Replay Utilisation in Sparse Reward Environments

In recent years, the widespread application of deep reinforcement learning (DRL) in autonomous systems has highlighted the importance of achieving high sample efficiency under sparse reward conditions. To improve sample efficiency in sparse reward environments, this paper proposes a reinforcement learning framework built upon the Soft Actor Critic architecture, which integrates multifaceted curiosity rewards (MCR) and adaptive experience replay utilisation (AERU) (MCR‐AERU SAC). MCR combines multi‐level intrinsic motivational signals, such as state prediction error and model uncertainty, to provide rich exploration incentives, encouraging the controlled entity to deviate from existing trajectories and discover new high‐reward behaviours. AERU dynamically adjusts the experience replay priorities based on posterior temporal difference error (TD error), focusing on utilising ‘partially successful’ transitions that are easily overlooked. The synergy between MCR and AERU enables the proposed framework to achieve an optimal balance between exploration and exploitation, significantly accelerating policy convergence and improving sample utilisation. Extensive experiments in complex dynamic training environments demonstrate that the proposed MCR‐AERU SAC algorithm achieves up to 1.41 times the early‐stage gain rate and a 55.56% improvement in task success rate compared to the HER‐SAC baseline, demonstrating superior sample efficiency and excellent robustness in large‐scale sparse reward environments.

Repetition as Reinforcement: Enhancing Sample Efficiency via Instant Episode Repetition in Reinforcement Learning

Instant Episode Repetition (IER) is introduced, a simple and novel mechanism that improves sample efficiency by immediately repeating action sequences from successful episodes during environment interaction by directly influences the data collection process.