Intelligent EfficientNetB7-based Surveillance Video Abnormality Classification Approach with Object Detection and Tracking using YOLOv9 with DeepSORT

Abstract

Video surveillance systems help in tracking and monitoring the real-time events and also anomalous activities. An automated object detection and tracking poses security concerns that minimize the reliance on human intervention. Recent deep learning models provides high productivity and accuracy in dealing with videos of different qualities, however, as surveillance videos have less resolution and poor visibility, more rigorous strategies are required for better tracking of anomaly events. This research introduces object detection and tracking model based on abnormal recognition, enabling better security in crowded areas. Initially, the required videos are gathered through public online databases. The collected videos are directly fed into the object detection and tracking module, where YOLOv9 with DeepSORT (Yv9-DSORT) model is designed to track the objects across the video frames. The detected and tracked frames of the object are passed to the abnormal classification stage, whereas the EfficientNetB7 model is employed to provide abnormality classification results. The classification model identifies complex spatio-temporal patterns and small variations in abnormal regions for threat detection. The developed approach precisely identifies the unusual events. The resultant classified outcomes are validated with the baseline models to ensure its effectiveness.