Reinforcement learning-based control for autonomous vehicle trajectory tracking under coupled physical-cyber disturbances

Abstract

This paper presents a reinforcement learning (RL)-based trajectory tracking control method for an autonomous vehicle (AV) under coupled road disturbances and denial-of-service (DoS) attacks. Unlike existing studies that treat physical-layer disturbances and network-layer attacks separately, we construct a unified augmented system integrating a seven-degree-of-freedom vehicle model with a stochastic DoS process. A task-oriented value function is designed by embedding DoS attack probability into the performance index, enabling the controller to adapt its strategy based on attack intensity. To improve learning efficiency, we propose a structured Q-learning approach that updates the optimal control and disturbance strategy online without requiring exact prior knowledge of the vehicle’s physical parameters. Simulation results demonstrate that the proposed method outperforms robust control and adaptive model predictive control in tracking accuracy under varying attack intensities, validating the effectiveness of the unified RL-based framework for AVs under coupled physical-cyber threats.