Infrared-radar heterogeneous fusion personnel perception technology based on spatio-temporal constraints in complex electromagnetic environments

Abstract

To address the challenges in electric power operation environments, such as substations and distribution rooms—where personnel intrusion detection and positioning are susceptible to equipment occlusion, drastic lighting variations, and strong electromagnetic interference, making it difficult for single sensors to balance stability and accuracy—this paper proposes a multimodal fusion perception method based on an infrared thermopile array and millimeter-wave radar. Leveraging the advantages of infrared thermal imaging in perceiving human thermal radiation, the proposed method utilizes an improved Ghost-NanoDet model to obtain the personnel position in the image plane, while incorporating distance and azimuth information provided by the millimeter-wave radar for spatial completion. To mitigate perception instability caused by thermal interference from power equipment and multipath effects from metal structures, a spatio-temporal joint fusion algorithm based on angular consistency and temporal continuity constraints is designed to achieve effective association between infrared vision and radar ranging information. Experimental results show that the proposed fusion method achieves 95.2 % detection success rate, 1.5 % false-alarm rate and 0.25 m positioning error under typical substation interference. Its robustness applies to conventional substation electromagnetic scenarios rather than extreme radiation that may damage sensors. This work provides an edge-side multimodal sensing solution for power-site personnel monitoring and references low-cost thermopile-radar fusion for intelligent perception.