LLM-Driven Automated Reward Design for Reinforcement Learning-Based Routing in LEO Satellite Networks

Abstract

Routing in Low Earth Orbit (LEO) satellite networks is challenging due to highly dynamic topologies and spatio-temporal network conditions. Reinforcement Learning (RL) has emerged as a promising approach for adaptive routing; however, its performance critically depends on reward function design, which must balance objectives such as goodput and end-to-end delay. In practice, reward design remains a complex manual process requiring significant domain expertise and extensive trial-and-error. Recent works have explored Large Language Models (LLMs) for automated reward design, but their application to highly dynamic systems such as LEO satellite networks remains largely unexplored. We propose LARGE, a framework that automates reward design for RL-based routing by combining LLM- driven generation with iterative simulator-in-the-loop evaluation. LARGE generates an initial reward from LLM prior knowledge and iteratively refines it using simulation feedback. This loop enables exploration of diverse reward formulations while aligning them with network objectives. Results show that LARGE improves reward quality within a few iterations through feedback-driven refinement. Across different backbones, the framework achieves performance comparable to an expert-designed baseline, with the best-performing configuration reaching goodput within approximately 3% of the baseline and slightly lower end-to-end delay, without manual reward engineering. These results indicate that effectiveness emerges from the iterative feedback-driven process enabled by LARGE, highlighting the potential of framework-driven LLM-in-the-loop optimization for RL-based routing in dynamic satellite networks.