Edge-Accelerated Cooperative Sensing Over CAV Networks: A Transformer-Based RL Approach

Abstract

Environment sensing plays an essential role in connected autonomous vehicles (CAVs) by enabling timely and accurate awareness of the surrounding environment. However, the performance of standalone sensing is constrained by sensing accuracy and computing resources of each individual CAV. In this paper, we propose a novel edge-accelerated cooperative sensing framework for CAV networks, in which LiDAR sensing data and computing resources from the roadside unit (RSU) and CAV are cooperatively fused to improve sensing accuracy and reduce task completion delay. The RSU makes online task placement decisions, in which each sensing task can be processed at the CAV or the RSU. Our objective is to design a task placement and resource allocation policy that minimizes the average task completion delay subject to the long-term sensing accuracy constraint. To this end, we first formulate the minimization problem as a constrained Markov decision process (CMDP) and then reformulate it as an MDP using the Lyapunov optimization technique. To solve this problem, we further propose a novel two-layer transformer-based reinforcement learning (TRL) algorithm, which can handle the time-varying state-action space issue in CAV environments. Specifically, in the outer layer, the task placement decision is determined by the TRL algorithm, which employs transformer blocks to constrain the state space and generate feasible actions via an action mask. In the inner layer, the optimal resource allocation decision is determined by a low-complexity greedy-based subroutine. Extensive simulation results using the autonomous driving simulator CARLA demonstrate that the proposed solution can reduce task completion delay by up to 39.7% compared with state-of-the-art benchmarks.