Federated reinforcement learning for energy-aware load balancing in edge-fog-cloud IoT continuum

Abstract

Energy efficiency remains a major challenge in deploying IoT systems, especially in scenarios requiring large numbers of devices while balancing computational requirements and operational lifetimes. This paper proposes a federated reinforcement learning framework for adaptive load balancing in the edge-fog-cloud continuum that optimizes energy efficiency and supports diverse quality of service requirements. The proposed framework addresses the limitations of traditional centralized machine learning approaches that require collecting sensitive operational information and transmitting it to cloud servers for centralized analysis. This increases the risk of privacy violations and introduces communication overheads that limit the responsiveness of IoT systems. The proposed framework employs a federated reinforcement learning approach, enabling edge nodes to collaboratively learn an optimal load-balancing policy without transmitting operational information. The proposed framework uses a context-aware reward function that optimizes multiple objectives based on temporal patterns, device energy levels, and application criticality. This enables the proposed framework to adapt its optimization objectives and balance energy efficiency and performance maximization. The proposed framework introduces a new action-space pruning mechanism that accelerates the optimization process by leveraging domain knowledge of possible load-balancing patterns. The proposed framework uses a distributed experience replay buffer to reduce trial-and-error in reinforcement learning. The proposed framework demonstrates its effectiveness in optimizing energy efficiency through a series of experiments in a real-world IoT environment and a centralized machine learning approach. The proposed framework demonstrates that distributed machine learning approaches can outperform centralized ones for optimizing energy efficiency in IoT systems.