Deep Learning-Driven Pose Extraction and Spatiotemporal Refinement for Human Detection and Distance Estimation on Railway Tracks with Thermal Imaging

Abstract

Human presence in rail track areas represents a critical safety risk, particularly under low-visibility conditions where RGB-based monitoring systems become unreliable. This paper presents a thermal-only railway monitoring framework for human detection, auxiliary pose-based skeletal representation, identity-preserving tracking, temporal refinement, track zone classification, and camera-to-human distance estimation. The proposed approach combines a YOLO-based human detector trained on annotated thermal railway images with a pretrained YOLO pose model used as an auxiliary module to obtain a reduced 13-keypoint skeletal representation for thermal-image interpretation. ByteTrack is used for identity association across frames, while Kalman filtering reduces frame-to-frame keypoint localization jitter and supports short-term trajectory continuity during temporary missed detections. Unlike fully learned monocular depth estimation, the proposed distance estimation method exploits rail track geometry as a physical scene reference. A YOLO-based rail track detection model extracts the rail region, after which rail line candidates are fitted, and the apparent rail track width is measured at the vertical position of the detected human. Distance is estimated using an inverse-perspective relationship based on the known railway gauge. Experiments on thermal video sequences acquired along the Niš–Prokuplje railway line demonstrate reliable human detection, stable tracking, effective track zone classification, and distance estimation consistent with real distance trends.