Toward Automatic Pavement-Condition Index Estimation: An Enhanced Multi-Sensor Deep Learning Framework for Comprehensive Pavement Distress Detection and Severity Assessment

Abstract

Timely and accurate pavement distress detection is essential for sustainable road maintenance. Traditional manual methods are costly, time-consuming, and prone to subjectivity. This study proposes a multi-sensor framework that integrates (Red-Green-Blue (RGB) imagery with low-cost depth sensing for automated detection, classification, and severity quantification of pavement distresses. The system identifies and categorizes 11 common distress types using convolutional neural networks and a sensor fusion strategy. Detected regions are projected onto three-dimensional point clouds to enable class-specific severity assessment. Pothole and rutting severity classification are examined as case studies. Using a low-cost Red Green Blue plus Depth (RGB-D) multi-sensor fusion framework, we report, for the first time, automated rutting severity classification integrated into Pavement-Condition Index (PCI) estimation as a proof-of-concept, alongside enhanced pothole severity quantification and classification. Results show high accuracy, with pothole severity correctly estimated in 96.2% of cases and rutting severity classified with 100% accuracy. The system also reliably detects shallow potholes as small as 15 mm in depth. By automating key measurements required for PCI estimation, this work lays a practical foundation for data-driven pavement evaluation. The proposed approach offers a scalable and cost-effective solution for infrastructure monitoring, reducing reliance on manual inspection and enhancing roadway management, while also demonstrating extensibility to other distress types.