Asynchronous Federated Reinforcement Learning for Adaptive Resource Slicing and Low-Latency Task Offloading in Heterogeneous 6G Edge Computing Networks

Abstract

The emerging paradigm of 6G wireless communication networks envisions ultra-reliable low-latency communication (URLLC), massive machine-type communications (mMTC), and pervasive edge computing intelligence. In heterogeneous mobile edge computing (MEC) networks, dynamically offloading compute-intensive tasks (e.g., augmented reality rendering, connected vehicular telemetry, autonomous robotic control) while orchestrating multi-tenant network slicing under time-varying channel conditions is an NP-hard stochastic optimization problem. Centralized reinforcement learning algorithms suffer from extreme communication overhead, severe backhaul congestion, and severe privacy vulnerabilities. Conversely, standard synchronous Federated Learning (FL) methods encounter severe 'straggler effects' caused by heterogeneous edge device processing capabilities. In this paper, we propose AF-EdgeRL, a novel Byzantine-resilient Asynchronous Federated Reinforcement Learning framework tailored for distributed resource allocation and dynamic task offloading. AF-EdgeRL deploys a distributed Proximal Policy Optimization (PPO) agent across edge servers and end-user devices, combined with a Staleness-Aware Adaptive Weight Aggregator (SAWA) that dynamically adjusts model update gradients based on hardware compute latency and channel state information (CSI). Furthermore, we establish theoretical convergence guarantees under non-convex reinforcement learning objectives. Evaluated on a high-fidelity 6G MEC simulator with real-world mobile mobility traces (Telecom Italia Milano dataset), AF-EdgeRL reduces end-to-end task execution latency by 41.2%, achieves 99.999% URLLC deadline compliance, and decreases edge energy consumption by 32.6% compared to state-of-the-art synchronous FedRL and centralized DRL baselines.