Deep Reinforcement Learning Driven Spectrum-Energy Joint Optimization

Abstract

Spectrum efficiency (SE) and energy efficiency (EE) are two fundamental problems in the wireless resource management that need to be jointly optimized. Deep reinforcement learning (DRL) allows near-optimal policy learning via continuous interaction with the environment, which is suitable in complex, dynamic, and high-dimensional network problems in comparison to traditional model-based optimization methods. This paper gives a detailed analysis of the DRL-Based spectrum energy joint optimization methods. To begin with, the problem statement and the determination of the major design aspects were described. Then, single-link and simple network models, multi-link and multi-user interference models, and the new architectures with reconfigurable intelligent surfaces (RIS) are introduced. Further comparative analysis is done based on algorithmic paradigms, optimization goals, scalability, and deployment feasibility. The existing limitations are then discussed, such as imperfect channel state information, computational complexity and signaling overhead, limited generalization and interpretability, and the absence of common evaluation standards. New directions in research including partially observable Markov decision process formulations, federated reinforcement learning, hierarchical decision-making, and 6G-oriented joint optimization frameworks are mentioned. This paper will help in not only achieving a deeper theoretical insight, but also practical application of effective resource allocation mechanisms in next-generation wireless systems.