reinforcement learning

PBFT-CG-MARL

PBFT-CG-MAPPO is a consensus-conditioned multi-agent reinforcement learning framework that embeds a Practical Byzantine Fault Tolerance (PBFT) three-phase commit protocol inside the CTDE-MAPPO training loop as an additive consensus loss with zero initialization, complemented by an entropy floor that prevents premature policy collapse. Across three cooperative environments (MPE spread, SMAClite 5m_vs_6m, VMAS UAV coverage) with five seeds each, PBFT-CG-MAPPO reduces cross-seed return variance by 1.9–47.4× compared to MAPPO and produces zero catastrophic seeds. Under Byzantine injection (f = 1), the PBFT quorum maintains stable consensus rates against random and adversarial attacks, where disabling the consensus layer causes up to 93% return degradation—demonstrating that consensus provides critical protection under active attacks while remaining non-interfering in clean environments. The experimental results yield three design principles for safe consensus conditioning in MARL: replace only dissenting actions (never overwrite consenting agents), condition via additive loss with zero initialization, and enforce an entropy floor. The framework transfers from 4-agent cooperative navigation to 12-agent permafrost monitoring without modification, scaling the f < n/3 tolerance bound automatically. The codebase includes six algorithm baselines (MAPPO, MADDPG, QMIX, CommNet, TarMAC), three Byzantine attack types, ablation studies, cross-environment evaluation, and publication-quality figure generation scripts.

Learning Among Learners: A Narrative Review of Multi-Agent Reinforcement Learning from Markov Games to Deep Emergent Play

Multi-agent reinforcement learning---the learning of behavior when the environment's other agents learn too---moved from Tan's independent learners and Littman's Markov games framework through the cooperative dynamics' analyses and the surveys' question to the deep era's communication, actor-critics, value decompositions, and the large-scale emergent play of Capture the Flag. This article presents a narrative review of that arc's canonical line: Tan's 1993 independent versus cooperative agents, Littman's 1994 Markov games, Claus and Boutilier's 1998 cooperative dynamics, Hu and Wellman's 1998 framework, Shoham, Powers, and Grenager's 2007 question, Busoniu, Babuska, and De Schutter's 2008 survey, Foerster and colleagues' 2016 learning to communicate, Lowe and colleagues' 2017 multi-agent actor-critic, Sunehag and colleagues' 2018 value-decomposition networks, Rashid and colleagues' 2018 QMIX, Jaderberg and colleagues' 2019 3D multiplayer Capture the Flag, and Hernandez-Leal, Kartal, and Taylor's 2019 survey and critique. The review is organized around three themes: the foundational frames, in which the Markov game's formalization and the non-stationarity's, the coordination's, and the equilibrium's problems defined the field's difficulties; the theory's question, in which the surveys asked what learning among learners is for; and the deep era, in which the communications, the centralized critics, the monotonic factorizations, and the population-scale play made the multi-agent learning practical. It is concluded that multi-agent reinforcement learning is the non-stationarity's discipline---and that its deep era turned the other learners' obstruction into the curriculum's engine.

Integrasi Teori Perilaku Belajar dan Rekonstruksi Penguatan dalam Pembelajaran Pendidikan Agama Islam

In educational psychology, the theory of student learning behavior is one of the most important topics. The diversity of theories on this subject demonstrates the numerous ways teachers can easily understand students’ conditions and master how to teach the lessons they will teach. The purpose of this study is to focus on explaining the nature, definition, and principles of learning, the theory of learning behavior, the relationship between learning and teaching, and the application of learning principles in Islamic Religious Education (PAI) learning. The method used in this study was a literature review with a qualitative approach. The research findings indicate that behavioral theories, particularly the Contiguity theory, Connectionism theory, Classical Conditioning theory, Operant Conditioning theory, and Social Learning theory, can be easily applied to various aspects of Islamic Education learning by creating conducive associations and positive reinforcement. In PAI learning, the application of learning principles is crucial to increasing learning effectiveness, particularly in achieving one of the goals of Islamic religious education, namely shaping student behavior in accordance with religious teachings.

Smartpath-Rpl: A Deep Reinforcement Learning-Based Optimized Routing Protocol For Mobile Low-Power And Lossy Networks

PBFT-CG-MARL

PBFT-CG-MAPPO is a consensus-conditioned multi-agent reinforcement learning framework that embeds a Practical Byzantine Fault Tolerance (PBFT) three-phase commit protocol inside the CTDE-MAPPO training loop as an additive consensus loss with zero initialization, complemented by an entropy floor that prevents premature policy collapse. Across three cooperative environments (MPE spread, SMAClite 5m_vs_6m, VMAS UAV coverage) with five seeds each, PBFT-CG-MAPPO reduces cross-seed return variance by 1.9–47.4× compared to MAPPO and produces zero catastrophic seeds. Under Byzantine injection (f = 1), the PBFT quorum maintains stable consensus rates against random and adversarial attacks, where disabling the consensus layer causes up to 93% return degradation—demonstrating that consensus provides critical protection under active attacks while remaining non-interfering in clean environments. The experimental results yield three design principles for safe consensus conditioning in MARL: replace only dissenting actions (never overwrite consenting agents), condition via additive loss with zero initialization, and enforce an entropy floor. The framework transfers from 4-agent cooperative navigation to 12-agent permafrost monitoring without modification, scaling the f < n/3 tolerance bound automatically. The codebase includes six algorithm baselines (MAPPO, MADDPG, QMIX, CommNet, TarMAC), three Byzantine attack types, ablation studies, cross-environment evaluation, and publication-quality figure generation scripts.

Learning by Watching: A Narrative Review of Imitation Learning from ALVINN to Generative Adversarial Imitation

Imitation learning---the learning of behavior from demonstrations instead of rewards---moved from Pomerleau's ALVINN driving network and Schaal's humanoid route through Ng and Russell's inverse reinforcement learning, Abbeel and Ng's apprenticeship learning, and Ziebart's maximum entropy to the robot learning from demonstration surveys, Ross's DAgger, Ho and Ermon's generative adversarial imitation, Finn's guided cost learning, and the algorithmic perspective's syntheses. This article presents a narrative review of that arc's canonical line: Pomerleau's 1989 ALVINN, Schaal's 1999 humanoid question, Ng and Russell's 2000 inverse RL, Abbeel and Ng's 2004 apprenticeship learning, Billard and colleagues's 2008 handbook chapter, Ziebart and colleagues's 2008 maximum entropy, Argall and colleagues's 2009 survey, Ross, Gordon, and Bagnell's 2011 DAgger, Ho and Ermon's 2016 GAIL, Finn and colleagues's 2016 guided cost learning, Hussein and colleagues's 2017 survey, and Osa and colleagues's 2018 algorithmic perspective. The review is organized around three themes: the foundations, in which the driving network's demonstrations, the humanoid's question, and the inverse reward's recovery defined the field's two programs; the demonstration's surveys, in which the robot programming's handbook and the LfD's survey systematized the practice; and the deep era, in which the DAgger's covariate correction, the adversarial's discrimination, and the algorithmic perspective's synthesis unified the field. It is concluded that imitation learning is the reward's workaround---and that its arc is the demonstrator's knowledge's transfer from the human's steering to the policy's distributions.

Learning Among Learners: A Narrative Review of Multi-Agent Reinforcement Learning from Markov Games to Deep Emergent Play

Multi-agent reinforcement learning---the learning of behavior when the environment's other agents learn too---moved from Tan's independent learners and Littman's Markov games framework through the cooperative dynamics' analyses and the surveys' question to the deep era's communication, actor-critics, value decompositions, and the large-scale emergent play of Capture the Flag. This article presents a narrative review of that arc's canonical line: Tan's 1993 independent versus cooperative agents, Littman's 1994 Markov games, Claus and Boutilier's 1998 cooperative dynamics, Hu and Wellman's 1998 framework, Shoham, Powers, and Grenager's 2007 question, Busoniu, Babuska, and De Schutter's 2008 survey, Foerster and colleagues' 2016 learning to communicate, Lowe and colleagues' 2017 multi-agent actor-critic, Sunehag and colleagues' 2018 value-decomposition networks, Rashid and colleagues' 2018 QMIX, Jaderberg and colleagues' 2019 3D multiplayer Capture the Flag, and Hernandez-Leal, Kartal, and Taylor's 2019 survey and critique. The review is organized around three themes: the foundational frames, in which the Markov game's formalization and the non-stationarity's, the coordination's, and the equilibrium's problems defined the field's difficulties; the theory's question, in which the surveys asked what learning among learners is for; and the deep era, in which the communications, the centralized critics, the monotonic factorizations, and the population-scale play made the multi-agent learning practical. It is concluded that multi-agent reinforcement learning is the non-stationarity's discipline---and that its deep era turned the other learners' obstruction into the curriculum's engine.

When to Parallelize Stochastic Exploration of Rare Rewards in Reinforcement Learning

International audience

Residual RL on a PSO-tuned Fuzzy Controller for Mobile Robot Trajectory Tracking

This paper presents a two-wheeled mobile robot trajectory-tracking controller combining a particle swarm optimization (PSO)-tuned fuzzy logic controller (FLC) with a residual reinforcement learning (RL) correction layer.PSO tuning reduces the global distance error by 35% and the integral absolute error by 44% over the initial FLC.The residual RL layer further reduces the global distance error by approximately 2.3% and improves cornering-region tracking by 3.9% in RMSE, 4.7% in IAE, and 5.2% in peak distance error.The proposed controller also reduces the global distance error by 41% and 66% relative to independently tuned PID and fuzzy-PID baselines.Trained across four trajectory families with a held-out test split, the generalized agent reduces the average test distance error by 18% relative to the tuned FLC baseline.These results show that a lightweight residual correction improves both accuracy and generalization while preserving the fuzzy controller's interpretability.

Explainable AI for Reinforcement Learning via Causal Reasoning

Reinforcement learning (RL) has achieved remarkable success in various domains, but its "black box" nature poses a significant challenge for real-world deployment. Understanding the rationale behind an RL agent's decisions is crucial for trust, debugging, and improving performance. This paper proposes a novel approach to explainable AI (XAI) within reinforcement learning by leveraging causal reasoning. We model the environment and the agent's policy using a causal Bayesian network. By performing inference through this network, we trace the causal chain of events leading to a specific action, providing a transparent explanation. This method moves beyond simply observing the agent's behavior to understanding the underlying reasons for its choices. The core of our approach lies in identifying and representing the causal relationships within the RL system, enabling us to dissect the decision-making process and ultimately build more robust and reliable RL agents. The proposed framework offers a significant step toward interpretable RL and addresses a critical limitation of current techniques. ---

Adaptive Neuron Topology Optimization

This paper introduces a novel approach to neural network design termed Adaptive Neuron Topology Optimization. The core concept revolves around dynamically adjusting the connectivity topology of a neural network based on real-time monitoring of neuron activity. Traditional neural network topology is static, often hindering optimal performance. This work proposes a system leveraging reinforcement learning to optimize the network's topology. Specifically, a reinforcement learning algorithm is employed to modify both connection weights and the connections themselves between neurons, creating a self-adapting topology. The system aims to improve learning efficiency and generalization capabilities by allowing the network to evolve its structure based on the data it is processing. The key innovation lies in the dynamic, data-driven adaptation of the network topology, moving away from pre-defined static architectures. The system's effectiveness is demonstrated through a theoretical framework outlining the core mechanisms and potential benefits.

Adaptive Neuron Topology Optimization

This paper introduces a novel approach to neural network design termed Adaptive Neuron Topology Optimization. The core concept revolves around dynamically adjusting the connectivity topology of a neural network based on real-time monitoring of neuron activity. Traditional neural network topology is static, often hindering optimal performance. This work proposes a system leveraging reinforcement learning to optimize the network's topology. Specifically, a reinforcement learning algorithm is employed to modify both connection weights and the connections themselves between neurons, creating a self-adapting topology. The system aims to improve learning efficiency and generalization capabilities by allowing the network to evolve its structure based on the data it is processing. The key innovation lies in the dynamic, data-driven adaptation of the network topology, moving away from pre-defined static architectures. The system's effectiveness is demonstrated through a theoretical framework outlining the core mechanisms and potential benefits.