Traffic sign recognition robustness against real-world naturalistic and weather perturbations

Abstract

Traffic sign recognition systems are critical for autonomous vehicle safety yet remain vulnerable to adversarial perturbations that exploit environmental conditions. Existing naturalistic attacks employ simplified approximations without modeling underlying physics, while defenses lack comprehensive evaluation across diverse architectures and attack types. This work formulates six physically-grounded adversarial attacks spanning naturalistic perturbations (shadow, light patch, obstruction) and weather conditions (fog, snow, frost), optimized via Prior-guided Bayesian Optimization for black-box scenarios. We evaluate nine architectures across three geographically diverse datasets, revealing that transformer-based models exhibit 20.0 percentage points (pp) lower average attack success rate than CNNs (41.5% vs 61.5%) with comparable clean accuracy across datasets. We introduce Robustness Score (RS) to quantify resilience across all attacks, with baseline ConvNeXt-Tiny achieving 60.9% RS compared to best CNN at 50.7%. DiffPure diffusion-based purification substantially improves robustness through test-time noise injection and reverse denoising, increasing RS by 29.8-37.3 pp. Post-defense, transformer architectures achieve 6-16% residual attack success rate compared to 15-39% for CNNs, with consistent effectiveness across datasets (variance ⪅ 2.1 pp). These findings establish that architectural choice critically impacts adversarial robustness and that diffusion-based purification provides practical test-time defense for deployed autonomous driving systems.