GSPM-MAAC: Multi-Agent Reinforcement Learning for Task Offloading in Vehicular Edge Computing Systems

Abstract

In vehicular edge computing (VEC), existing task offloading approaches enable mobile vehicles (MVs) to offer ultra-low latency services for computation-intensive tasks. However, centralized offloading strategies are impractical due to MVs’ limited communication range and high mobility, requiring autonomous cooperation to enhance performance. Furthermore, existing approaches often overfit the training environment, performing poorly in test environments where other MVs behave differently. This paper proposes a Graph and Sorted Policy Memory Multi-Agent Actor-Critic (GSPM-MAAC) based offloading solution. First, the communication architecture among MVs is conceptualized as a learnable communication graph. Further, this problem is defined as the objective of identifying the communication graph while allowing the architecture parameters to be updated normally, thereby requiring a bi-level optimization procedure. Moreover, by introducing the attention mechanism and bi-level optimization method, the communication graph structure and architectural parameters are jointly optimized end-to-end. Additionally, a sorted policy memory (SPM) algorithm is introduced. Policies are ranked based on training episode rewards, and MVs randomly select policies as behavior policies to collect diverse multi-MV interaction data, enhancing training sample diversity. The experiments show that GSPM-MAAC improves task completion rates, reduces average delay, and can quickly be transferred to new VEC scenarios.