Cross temporal adaptive feature fusion for intelligent multi class surveillance crime recognition

Abstract

The increasing deployment of surveillance cameras has created a need for automated systems capable of recognizing diverse criminal activities from continuous video streams. Conventional approaches often rely on frame-level analysis or isolated temporal modeling, limiting their ability to capture subtle motion patterns, prolonged behavioral evolution, and person–object interactions. To address these limitations, this study proposes a novel Cross-Temporal Attention-Guided Adaptive Crime Recognition Network (CTAG-CRNet) for multi-class crime recognition in surveillance videos. The framework employs a convolutional neural network (CNN) for spatial feature extraction and introduces a Multi-Scale Spatial Crime Attention (MSSCA) mechanism to enhance fine-grained and contextual cues associated with weapons, human interactions, body movements, and scene activity. The enhanced representations are processed through complementary ConvLSTM and Long Short-Term Memory-based Convolutional Recurrent Network (LRCN) pathways to capture short-term motion dynamics and long-term behavioral evolution, respectively. Unlike conventional independent or direct-concatenation strategies, ACTAF-Net incorporates Cross-Temporal Interaction Attention (CTIA) to enable bidirectional information exchange between the temporal representations. Subsequently, Adaptive Temporal Reliability-Gated Fusion (ATRGF) dynamically determines the contribution of short- and long-term features and generates a discriminative fused representation for classification. The framework targets four crime categories: gun-related threats, passenger aggression, petty theft, and vandalism. Comprehensive evaluation includes comparative analysis, temporal-window sensitivity, component-wise ablation, class-wise performance, and computational efficiency. The proposed ACTAF-Net provides an adaptive spatiotemporal representation by jointly exploiting multi-scale spatial cues, complementary temporal dependencies, cross-temporal interactions, and reliability-aware feature fusion for robust surveillance-based crime recognition.