Evaluating the Robustness of Segmentation Models against Adversarial Patches in Off-Road Environments

Abstract

Semantic Segmentation (SS) is critical for autonomous vehicles to navigate off-road environments by identifying drivable terrain. Although models like ResNet34+UNet and EfficientViT have been proposed for these tasks, their susceptibility to localized adversarial patches in unstructured environments remains under-researched. This paper presents a comprehensive robustness evaluation of six real-time SS architectures, including the state-of-the-art YOLOv11 and YOLOv12 segmentation variants against five diverse adversarial patch schemes. Our experiments, conducted on a modified YCOR dataset, demonstrate that EfficientViT is the most resilient architecture, maintaining high accuracy with minimal performance degradation. In contrast, single-stage models like YOLOv11n-seg exhibit significant vulnerability, with pixel accuracy drops reaching 26.55%. We also show how decreases in overall segmentation accuracy impact the segmentation models’ ability to discern traversable terrain from non-traversable terrain.