Secure Hierarchical RSMA Against Multiple External and Internal Eavesdroppers

Abstract

Existing secure rate splitting multiple access (RSMA) designs are typically tailored to either external or internal eavesdropping and may be ineffective in mixed scenarios where both coexist. To address mixed internal and external eavesdropping, we propose a novel secure hierarchical RSMA (HRSMA) framework. In this framework, all common streams can act as artificial noise (AN) to mitigate external eavesdropping; more importantly, the trusted user (T-UE) group common stream simultaneously serves as AN to suppress internal eavesdropping by the untrusted served users (U-UEs). To ensure the security, our approach aims to maximize the secrecy sum rate of all T-UEs and solve the resulting highly non-convex problem via an iterative algorithm combining majorization–minimization (MM) and semidefinite relaxation (SDR). Simulation results demonstrate that the HRSMA achieves higher secrecy rates than conventional baselines.