YOLO-Based Scene Recognition via Object Co-Occurrence for the Visually Impaired

Abstract

Individuals with visual impairments face persistent difficulties in perceiving the context of their surroundings, since most assistive tools report isolated objects rather than the overall scene. This paper presents an image-based scene recognition framework, evaluated offline as a laptop simulation on static images, that combines deep learning-based object detection with a transparent, rule-based classification stage. YOLOv8m and YOLOv11m detectors were trained to recognize 16 everyday object classes associated with three scene categories: kitchen, classroom, and roadside. Detected objects are mapped to scene labels using an object co-occurrence table and predefined keyword sets, which makes the classification stage interpretable and easily extensible without retraining. Two datasets were used to quantify the effect of data quality: an initial set of 920 images with basic augmentation and an enhanced set of 6,170 images with targeted class balancing and augmentation covering flipping, rotation, hue, brightness, and blur. Across all hyperparameter configurations, YOLOv11m outperformed YOLOv8m, achieving 0.882 accuracy, 0.838 precision, 0.843 recall, and 0.8997 mAP@50 at 60 epochs with a batch size of 16 and a learning rate of 0.001, compared with 0.856 accuracy and 0.8405 mAP@50 for the best YOLOv8m configuration. The enhanced dataset, evaluated with the higher-capacity medium-variant models, improved accuracy by approximately 9 percentage points over the small-variant baseline on the initial dataset; because dataset size and model capacity were increased together, this gain reflects their combined effect rather than dataset enhancement in isolation. On the initial dataset, a manual scene-level verification of all 317 images of the independent evaluation set confirmed end-to-end pipeline accuracy of 0.78–0.79. The results indicate that pairing a modern single-stage detector with lightweight co-occurrence reasoning provides accurate and explainable scene-level feedback suitable for assistive applications.