A Reinforcement Learning-Based Scheduling Scheme for FSO and RF Hybrid Satellite-to-Ground Transmission Systems

Abstract

Low Earth orbit (LEO) satellite-terrestrial communication systems grapple with significant challenges posed by their inherent dynamism and substantial transmission delays. To address these critical issues, this paper proposes a novel hybrid-medium transmission optimization framework that leverages high-altitude platforms (HAPs) as relays. Our primary objective is to minimize end-to-end system delay through the joint optimization of transmission mode selection and wireless communication resource allocation. The resulting joint optimization problem is formulated as a computationally intractable mixed-integer nonlinear programming (MINLP). We present a hierarchical solution strategy to tackle this complexity. Firstly, Lagrangian optimization is employed to analytically derive the intrinsic coupling between resource allocation and transmission mode selection, thereby simplifying the problem into a sequential decision-making process. This sequential problem is subsequently framed as a Markov decision process (MDP), enabling the design of a deep reinforcement learning (DRL) agent tasked with dynamically learning the optimal transmission mode selection policy. By maximizing cumulative long-term rewards, our DRL-based approach effectively reduces overall system delay, unlocking enhanced performance potential for future 6G networks.