Stability-Aware and Feasibility-Sensitive Aggregation in Federated Reinforcement Learning for Edge-IoT Systems: A Review

Abstract

Federated Reinforcement Learning (FRL) provides a useful basis for distributed policy learning in Edge-IoT systems, where clients interact with local environments without transferring raw operational data to a central server. Yet aggregation becomes difficult when clients operate under different transition dynamics, workloads, resource capacities, communication conditions, and operational constraints. In these settings, local policy updates may not differ only in magnitude or direction; they may also differ in stability, reliability, resource support, and operational feasibility. Conventional averaging is therefore limited, since it does not distinguish stable and feasible updates from unstable or constraint-violating ones. This paper examines aggregation stability and feasibility-sensitive aggregation in FRL for Edge-IoT systems. It reviews and synthesizes related literature across four connected streams: heterogeneous Federated Learning, FRL-based edge decision-making, constrained and safe Reinforcement Learning, and adaptive or reliability-aware aggregation. The reviewed studies are analyzed through six dimensions: learning paradigm, type of heterogeneity, role of policy learning, treatment of operational constraints, aggregation strategy, and whether local feasibility signals influence global aggregation weights. The analysis indicates that existing studies provide valuable foundations, but they usually treat heterogeneity, constraint handling, and aggregation adaptation as separate concerns. The paper identifies a need for aggregation mechanisms that jointly account for update stability, update reliability, resource availability, and constraint feasibility. It positions aggregation as adaptive client influence regulation rather than passive averaging in future Edge-IoT FRL systems.