2020· International Journal of Artificial Intelligence & Digital Transformation· Vol 3, pp. 01-13· 0 citations
TL;DR
This work reviews pre-2019 XRL approaches, categorizing them into policy explanation, reward decomposition, model transparency, and post-hoc interpretability methods, and proposes a framework that combines interpretable policies, surrogate models, attention mechanisms, and visualization techniques to enhance transparency without significantly reducing performance.
Abstract
Reinforcement Learning (RL) is widely used for solving sequential decision-making problems, enabling agents to learn optimal actions through interaction with dynamic environments. However, many RL models function as “black boxes,” making their decisions difficult to interpret—an issue that is especially critical in safety-sensitive domains like healthcare, finance, and autonomous systems. Explainable Reinforcement Learning (XRL) addresses this challenge by providing human-understandable insights into agent behavior, policy decisions, and reward structures. This work reviews pre-2019 XRL approaches, categorizing them into policy explanation, reward decomposition, model transparency, and post-hoc interpretability methods. It highlights the trade-off between performance and interpretability, particularly in complex, high-dimensional environments. A framework is proposed that combines interpretable policies, surrogate models, attention mechanisms, and visualization techniques to enhance transparency without significantly reducing performance. Evaluation metrics such as fidelity, comprehensibility, and consistency are used to assess explanation quality. The analysis shows that hybrid approaches—combining inherent interpretability with post-hoc explanations—offer the best balance between accuracy and transparency. Overall, XRL is essential for building trust in automated systems, with future research focusing on standardized evaluation methods and human-in-the-loop learning.
Autonomous robotic systems are increasingly deployed in industrial automation, healthcare, logistics, agriculture, defense, and intelligent transportation, where they must make complex decisions in dynamic environments. Reinforcement Learning (RL) enables robots to learn optimal actions through interaction with their environment, but most deep RL models function as black boxes, limiting transparency and trust in safety-critical applications. This paper proposes an Explainable Reinforcement Learning for Autonomous Robotic Decision Making (XORL) framework that integrates reinforcement learning with Explainable AI (XAI) to improve decision interpretability. The framework combines multimodal sensor data, policy optimization, confidence estimation, reward decomposition, policy visualization, and decision traceability to generate understandable explanations for robotic actions. It evaluates performance using metrics such as navigation success, obstacle avoidance, learning stability, computational efficiency, explanation consistency, and reliability. Experimental results demonstrate that XORL enhances decision transparency, operator trust, safety awareness, and autonomous task performance while maintaining competitive learning efficiency, supporting the development of trustworthy and human-centric autonomous robotic systems.
Seshagiri N· International Journal of Int...· 0 citations
Explainable Reinforcement Learning (XRL) seeks to make Reinforcement Learning (RL) policies more transparent and interpretable, a key requirement in safety-critical and human-centric scenarios. However, it is mostly based on user studies, thus targeting the needs of a specific audience and lacking shared evaluation metrics. On the other hand, logic-based approaches within eXplainable Artificial Intelligence (XAI) provide compact, human-readable abstractions of decision-making. However, the systematic quantification of the explainability degree of logical representations remains an open problem. This work aims to advance the state of the art in XRL by introducing objective and planning-oriented metrics for policy explainability in RL settings. At the same time, it contributes to the field of logic for XAI by providing a principled way to quantify the explainability of logical rules, moving beyond common-sense assessments and simple propositional fragments. We employ Inductive Logic Programming (ILP) to extract symbolic representations of RL policies and define a novel set of explainability metrics, including activation rate, feature coverage, syntactic distance and semantic distance. These metrics quantify alignment between symbolic rules and agent behavior, the role of features in decision-making, and the evolution of policies during training and across agents in single and multi-agent RL. Experiments across different RL domains show that the proposed metrics highlight action-specific learning dynamics beyond global return, provide fine-grained insights into domain features beyond classical approaches for global feature importance estimation, and uncover coordination, specialization, and adaptation patterns in MARL. Moreover, they provide crucial insights for the transfer and generalization of action-specific policies.
Celeste Veronese, Edoardo Zorzi, Daniele Meli et al.· 0 citations
Reinforcement Learning (RL) has been extensively used in autonomous and self-* systems, but RL policies, especially deep RL ones relying on neural networks, lack transparency and are difficult to understand. This can lead to diminished user trust, and makes for a more challenging verification of systems. To address this challenge, this paper introduces Explanations using Alternative Realities for Reinforcement Learning (EARL), a Python library to produce counterfactual explanations in RL settings. This library allows the user to produce explanations by exploring What-if scenarios to clarify agent behavior by comparing possible outcomes. Counterfactual explanations have been shown to be intuitive and user-friendly in psychology research, but have only recently been explored in RL, with existing implementations usually limited to toy examples and benchmarks. EARL supports counterfactual explanation generation in realistic RL-based self-adaptive systems. To demonstrate its applicability, we demonstrate its use in a simulation of CitiBikes, a self-adaptive bike-sharing system, and we provide evaluations showing how it performs in real applications.
Jasmina Gajcin, Juan C. Rosero, Ivana Dusparic· 0 citations
Reinforcement learning (RL) provides a framework for sequential decision making under explicit objectives. In its classical form, RL studies how an agent should act to maximise long-term reward in a dynamic environment. In richer settings, the problem extends beyond a single agent and fixed environment: intelligent behavior may require strategic interaction, adaptation to uncertainty, and reasoning over high-dimensional worlds. This thesis studies RL from two perspectives: algorithms in games and RL in the era of foundation models. The first part focuses on multi-agent RL in games. It examines how incentives, policies, and equilibrium concepts interact in competitive and general-sum environments, spanning two-player zero-sum games, large-scale video games, and multi-player settings with general structure. These works investigate learning in multi-agent systems and the behavior of RL methods in interactive environments. The second part studies RL with generative and foundation models, motivated by the idea that prior knowledge can enrich sequential decision making. Pretrained generative models and learned world models serve as representation tools and structured priors for planning, control, and policy optimization. The thesis develops diffusion-based world models, investigates RL for efficient video generation, explores generative models as policy classes, and studies interactive video world models in which actions shape future observations. It also addresses long-horizon modeling through architectures with memory. Together, these contributions present a unified view of RL as objective-driven adaptation in complex sequential domains. From strategic games to generative world models, the thesis highlights how RL connects decision making, environment modeling, and emerging foundation-model capabilities, offering a broader perspective on the principles underlying intelligent behavior.
Reinforcement Learning (RL) offers significant potential for solving complex, sequential decision-making problems in supply chain management, such as dynamic inventory control. However, the adoption of state-of-the-art deep RL methods is severely hampered by their "black box" nature, as the resulting policies are notoriously opaque and difficult to trust in high-stakes industrial environments. To address this critical interpretability-performance gap, we introduce Structured Policy Reinforcement Learning (SPRL), a novel framework that embeds transparency directly into the agent's learning process. SPRL uses a decoupled architecture to distill value estimates from a powerful Q-learning agent into an inherently interpretable Decision Tree (DT). Crucially, the entire process is governed by a hard operational constraint (Inventory Position Cap, [Formula: see text]), which acts as an operational guardrail to stabilize training and enforce lean policies. We demonstrate that the optimized SPRL-DT policy achieves highly efficient combined performance, realizing a mean total cost of [Formula: see text] (a 45% reduction compared to the constrained DQN baseline) while maintaining an excellent service level of over 91%. This performance is achieved with a low complexity structure ([Formula: see text] nodes), offering a highly transparent and verifiable solution. Our results validate that SPRL enables the development of trustworthy autonomous systems, paving the way for the practical deployment of RL in real-world supply chain operations.
Embodied planning requires agents to make coherent multi-step decisions based on dynamic visual observations and verbal goals. While recent vision-language models (VLMs) excel at static perception tasks, they struggle in interactive environments. Reinforcement learning (RL) offers a natural way to address this limitation, yet online RL approaches suffer from costly interaction and sparse rewards in embodied settings. This paper introduces ORBIT , an O n-policy R einforcement fine-tuning (RFT) framework with offline rewards for Em B od I ed T ask Planning, that preserves the generalization benefits of RFT while addressing the challenges of costly interaction and sparse rewards, supported by solid theoretical guarantees. Our approach is evaluated on EmbodiedBench, a recent benchmark for interactive embodied tasks, covering both in-domain and out-of-domain scenarios. Experimental results show that ORBIT achieves SOTA performance on EB-ALFRED, outper-forming all closed-source and online-RL-based methods, while being substantially more effi-cient in training speed and computational cost, remaining robust to sub-optimal expert trajectories, and exhibiting strong generalization to unseen environments. We released all code and data at https://github.com/mail-taii/Reinforced-Reasoning-for-Embodied-Planning
Di Wu, Jiaxin Fan, Chloe Gu et al.· Annual Meeting of the Associ...· 0 citations