Robust beamforming design for RIS-UAV assisted two-layer RSMA systems with hardware impairments

Abstract

This paper proposes a robust beamforming optimization algorithm for reconfigurable intelligent surface (RIS)-unmanned aerial vehicle (UAV) assisted two-layer rate-splitting multiple access (RSMA) system under imperfect successive interference cancellation (SIC). Firstly, fractional programming (FP) is employed to transform the non-convex system optimization problem into a tractable form. Secondly, an alternating optimization (AO) framework is introduced to decouple and iteratively optimize the active and passive beamforming. Finally, a riemannian manifold alternating direction method of multipliers (RM-ADMM) approach is proposed to tackle RIS phase shift constant-modulus constraints effectively, enhancing convergence performance and solution accuracy under unit-modulus constraints. Simulation results demonstrate that the proposed algorithm converges stably and effectively improves the system sum rate through the joint optimization of the base station (BS) active beamforming, RIS passive beamforming and two-layer rate-splitting multiple access (RSMA). The proposed scheme achieves superior performance with increasing BS antennas, transmit power and RIS reflecting elements, while maintaining stronger robustness than single-layer RSMA and random RIS phase-shift schemes under imperfect successive interference cancellation (SIC).