Vehicle Detection and Classification with Compact Sensor Technologies and Convolutional Neural Networks

Abstract

Unattended ground sensors (UGSs) are passive systems used to detect and classify nearby activity such as the movement of military vehicles or personnel. However, UGS performance is limited in complex environments. This work explores the application of deep learning to classify vehicles as either ‘heavy’ or ‘light’ using seismic, acoustic, and magnetic sensor data collected at varying distances from a road. Two approaches were evaluated: one using features from Short-Time Fourier Transforms (STFTs), and another using convolutional neural networks trained on time series data. Models were also tested under data partitioning strategies involving either spatial or temporal variation. Data fusion was examined by comparing models that utilized all sensor modalities to those using a single modality. Layer-wise Relevance Propagation was applied to identify informative modalities and frequency components. Models utilizing frequency features outperformed time series-based models, with a 3% accuracy difference between the top performers. A 6% higher accuracy observed with spatial-only data partitioning compared with temporal partitioning suggests an overestimation of performance, highlighting the need to incorporate temporally distinct data. The STFT-based fusion model achieved the highest accuracy. Acoustic data, particularly in the frequency range below 100 Hz, was most relevant. Overall, the results demonstrate the effectiveness of transformation of the time series data to the frequency domain and the importance of careful data partitioning for reliable performance assessment. Future work should explore alternative, finer-tuned neural networks.