Dimension-Independent Multi-Agent DRL for Multi-Cell Interference Mitigation

Abstract

Multi-agent deep reinforcement learning (DRL) offers a promising framework for inter-cell interference mitigation in multi-cell networks. In such networks, each cell is associated with an agent that learns from its local environment to maximize a reward, such as spectral efficiency. To effectively mitigate inter-cell interference, agents typically share model weights or local experiences with one another or with a central node. However, the exchange of such information incurs significant communication overhead in each communication round between the central node and the individual agents, posing a major bottleneck to efficient multi-agent DRL-based inter-cell interference mitigation. This paper presents a novel dimension-independent multi-agent DRL algorithm for multi-cell interference mitigation. By leveraging zeroth-order optimization, the proposed algorithm reduces the communication overhead from <inline-formula> <tex-math notation="LaTeX">$\mathcal {O}(d)$ </tex-math></inline-formula> to <inline-formula> <tex-math notation="LaTeX">$\mathcal {O}(1)$ </tex-math></inline-formula>, where <inline-formula> <tex-math notation="LaTeX">$d$ </tex-math></inline-formula> denotes the shared information dimension. This is achieved by exchanging only a constant number of scalar values between the central node and the agents in each communication round, independent of the dimension <inline-formula> <tex-math notation="LaTeX">$d$ </tex-math></inline-formula> of the shared weights or experiences. The proposed algorithm is evaluated on millimeter-wave networks with varying numbers of cells, demonstrating its effectiveness for interference mitigation. Specifically, under universal frequency reuse, the total sum-rate increases almost linearly with the number of cells. Simulation results show that the proposed algorithm effectively mitigates interference and maximizes spectral efficiency in line-of-sight (LoS), non-LoS, and mixed environments, while significantly reducing communication overhead.