Maneuver Sequence Coverage of Scenarios for Autonomous Driving System
TL;DR
A set of scenario-level coverage metrics for ADS testing that characterize temporal combinations of driving maneuvers are introduced, highlighting that incorporating temporal maneuver sequences yields a more rigorous and sensitive measure of test adequacy for autonomous driving systems.
Abstract
Autonomous Driving Systems (ADS), such as Apollo and Autoware, are complex cyber-physical platforms that fuse perception, planning, and control to ensure safe vehicle operation in dynamic environments. Achieving high safety assurance demands rigorous testing across diverse and challenging driving conditions. However, due to the inherent incompleteness of scenario-based testing, it remains essential to evaluate how effectively scenario datasets exercise system behavior and expose potential failures. This paper introduces a set of scenario-level coverage metrics for ADS testing that characterize temporal combinations of driving maneuvers. Our key insight is that failures often emerge from specific sequences of maneuvers rather than from individual actions in isolation. Using a large-scale scenario dataset, we assess the capability of these metrics to reveal system violations and unsafe behaviors. Experimental results show that 4-way sequential coverage achieves a 100% detection rate for violations and threats, significantly outperforming both 4-way combinatorial coverage (35%) and a state-of-the-art metric, ComOpt (≈ 30%). Overall, our findings highlight that incorporating temporal maneuver sequences yields a more rigorous and sensitive measure of test adequacy for autonomous driving systems.