Lightweight Multi-Scale Transformer for Real-Time Railway Surface Defect Detection on Inspection Vehicles

Abstract

Railway surface defect detection is essential for condition-based maintenance and structural health monitoring of rail infrastructure. With the deployment of high-resolution cameras on inspection vehicles, detectors must localize small and diverse defects in real time under strict on-board computational constraints. Existing convolutional neural network (CNN) based methods often lack sufficient long-range context along the rail, whereas recent transformer-based architectures are typically too heavy for embedded deployment. This paper proposes a Lightweight Multi-Scale Transformer (LMST) tailored to real-time railway surface defect detection on inspection vehicles. LMST combines rail-oriented tokenization, a hierarchical multi-scale transformer encoder with rail-aligned windowed self-attention, and a defect-aware gated fusion module feeding a lightweight dense prediction head. Experiments on a public rail surface defect benchmark and additional inspection-vehicle imagery show that LMST achieves competitive or improved average precision compared with strong CNN and transformer baselines, while maintaining real-time throughput on industrial GPUs and clearly enhancing the detection of small and elongated defects under challenging field conditions.