A novel deep reinforcement learning-based cell selection algorithm for efficient mobility management in 6G networks

Energy-Efficient Cooperative Data Offloading in Cellular Networks Using Reinforcement Learning

To address the growing need for wireless communications energy efficiency, this paper proposes a new multi-agent reinforcement learning (MARL) approach to cooperative data offloading in heterogeneous cellular networks. This research is among the first to employ MARL to this extent, and it offers an end-to-end solution that combines cellular, Wi-Fi, and device-to-device (D2D) communications and considers practical network environments like user mobility and channel conditions. We formulate the offloading problem as a Markov Decision Process (MDP) with correct models of energy consumption and network conditions. The deep Q-network (DQN)-based MARL algorithm allows user equipment (UEs) to learn collaborative strategies for optimizing overall energy consumption and timely offloading of data. Simulations compare MARL against greedy, random, and independent Q-learning baselines in low and high mobility regimes. Experiments show that MARL saves energy by as much as 40% over random offloading and 16.6% over greedy offloading, as well as enhancing average delay, throughput, and fairness. Convergence of the learning rate of the algorithm is within 1000 episodes, and sensitivity analyses confirm its performance across a range of user density and data size settings. Furthermore, the MARL framework accommodates dynamic network conditions and provides an adaptable solution for network operators to maximize performance and sustainability for existing and future wireless networks. The suggested MARL framework still performs better in terms of delay, throughput, and fairness, while at the same time catapulting energy savings over the greedy offloading approach by 9.4% to 12.8% under the very realistic 3GPP-compliant HARQ, adaptive modulation, standardized power control, and urban SLAW mobility models settings. With the extended state/action spaces and realistic 3GPP+SLAW conditions, MARL has an energy savings of 11.7%–14.2% over greedy offloading while still having the leading delay, throughput, and fairness.

Deep Reinforcement Learning for Handover Optimization in 5G Networks

Ultra-dense 5G networks require advanced traffic steering to maintain performance and balance load amid growing user and base station (gNB) densities. Traditional heuristics such as nearest-base-station and Signal-to-Interference-plus-Noise Ratio (SINR)-based selection provide simple solutions but struggle to adapt to dynamic user mobility, diverse traffic, and fluctuating radio conditions at the mobility-control level, often leading to inefficient handovers and degraded network quality. We propose a deep reinforcement learning (DRL) framework to dynamically tune a global handover hysteresis margin that governs handover triggering decisions, optimizing handover success, reducing failures, and enhancing throughput and fairness. Implemented in Python using Stable Baselines3 and NumPy, our custom simulation environment models key mobility-related 5G dynamics at a high level, including user mobility, pathloss-based signal degradation, and interference. We evaluate DRL agents-Deep Q-Network (DQN) and Proximal Policy Optimization (PPO)-against heuristic and hysteresis-based baselines. Results show that DRL-based hysteresis optimization provides strong and robust performance under the considered ultra-dense mobility conditions in handover success rate, average SINR, throughput, and fairness, with PPO demonstrating the most consistent behavior across configurations. This work offers a reproducible simulation framework for further research into adaptive mobility management.

A Hybrid Artificial Intelligence Framework for Reliable and Seamless Vertical Handover in Next-Generation Heterogeneous Networks

AP Selection and Power Control for Personalized Cell-Free Massive MIMO: Graph-Embedded Reinforcement Learning Approach

Sixth-generation (6G) mobile communication poses unprecedented challenges for resource scheduling under personalized demands. Cell-free massive multiple-input multiple-output (CF-mMIMO), with its user-centric characteristics, has emerged as a key technology for satisfying personalized demands. However, faced with heterogeneous quality-of-service (QoS) requirements, existing reinforcement learning schemes are constrained by partial observability, making it difficult to balance overall system performance and personalized demands. Consequently, we propose a graph-embedded multi-agent deep deterministic policy gradient (G-MADDPG) scheme. Guided by personalized demands, proposed G-MADDPG formulates a maximization problem for system weighted sum spectral efficiency and introduces differentiated QoS penalties. In addition, graph neural networks (GNNs) are embedded into the policy learning and value estimation processes of reinforcement learning, endowing agents with enhanced structural reception and cooperative capabilities. Simulation results demonstrate that proposed G-MADDPG scheme outperforms existing benchmark schemes in both convergence speed and performance evaluation.

DEEP REINFORCEMENT LEARNING-BASED DYNAMIC SPECTRUM ACCESS FOR 6G HETEROGENEOUS COGNITIVE RADIO NETWORKS

The proliferation of heterogeneous radio access technologies in sixth-generation (6G) wireless networks demands a fundamental rethinking of spectrum management strategies. Traditional spectrum sensing approaches, designed for relatively static channel conditions, are inadequate for the dynamic, interference-rich environments that characterize 6G deployments spanning sub-6 GHz, millimetre-wave, and terahertz bands simultaneously. This paper proposes a Deep Reinforcement Learning (DRL)-based framework for dynamic spectrum access in 6G heterogeneous Cognitive Radio Networks (Het-CRNs), wherein secondary users (SUs) learn optimal channel selection policies through direct interaction with the radio environment, without requiring explicit statistical channel models. Specifically, a Double Deep Q-Network (DDQN) architecture is adopted, augmented with a prioritized experience replay mechanism that accelerates policy convergence under non-stationary channel conditions. The proposed agent observes a composite state space encoding instantaneous channel occupancy, signal-to-interference-plus-noise ratio (SINR), primary user (PU) activity patterns, and residual energy levels, and selects actions that jointly optimize spectrum utilization efficiency, interference avoidance, and energy consumption. Simulation experiments conducted over a heterogeneous network topology with four primary users and eight secondary users demonstrate that the proposed DDQN-based scheme achieves a throughput gain of approximately 34% over conventional energy detection-based sensing, reduces interference to primary users by 61%, and attains a detection probability of 0.94 at a false alarm rate of 0.05. These results confirm the practical viability of DRL as a spectrum management backbone for next-generation cognitive radio systems.

AI-Driven Mobility Management in 5G and 6G Wireless Networks: A Survey

Next-generation wireless systems are becoming increasingly complex, and there is a growing need for intelligent mobility management mechanisms that can ensure service continuity while making efficient use of network resources. In 5G and future 6G networks, dense small-cell deployments, heterogeneous architectures, and highly mobile users mean that frequent handovers (HOs), uneven traffic distribution, and variable network conditions often lead to degraded user experience, higher signalling overhead, and inefficient use of resources. Because user movement continuously redistributes traffic across cells, effective mobility management is inseparable from load balancing, and the HO process serves as the primary mechanism through which the network manages both. Recent advances in artificial intelligence (AI), particularly machine learning (ML) and deep learning (DL), offer an opportunity to transform mobility management from reactive to predictive, since data-driven solutions can forecast user movement, fine-tune HO execution, and dynamically allocate radio resources. This paper presents a comprehensive survey of AI-enabled mobility management strategies for 5G, Beyond 5G, and upcoming 6G networks, with particular attention to HO optimization and load balancing. The surveyed literature is organized around the complete lifecycle of AI-enabled mobility management, from mobility prediction and HO decision-making through parameter optimization and execution to KPI monitoring and model updating. This structure is used to classify existing frameworks according to their architectures, learning approaches, and optimization goals. The survey then examines how intelligent HO schemes address critical issues such as load balancing, interference mitigation, connection reliability, and quality-of-service maintenance, and compares conventional and AI-based methods against standardized key performance indicators for mobility robustness, resource efficiency, and service continuity. Finally, the paper discusses unresolved problems and emerging trends, including federated learning, multi-connectivity, and non-terrestrial integration, that will shape the evolution of autonomous mobility management solutions for future wireless networks.