Fine-Grained Visual Ambiguity Detection in Navigator for Visually Challenged People
TL;DR
The proposed Fine-Grained Visual Hybrid Deep Learning Model for Visually Challenged People (FGVHDL) invokes a parallel object detection stream and semantic refinement network to estimate the visual ambiguities.
Abstract
An assistive camera system that uses lightweight processing to guide visually challenged and low-vision users around everyday obstacles in real time with ambiguity. There is a need to identify the visual ambiguities between geometric and photometric, texture and elevation. A detector requisite to differentiate the fine-grained obstacles such as potholes vs shadows, drain covers vs flat area in indoor and outdoor environment conditions. Single- and two-stage detectors applied on edge devices to balance between speed and spatial-semantics quality for reducing missed obstacles and latency problems. The proposed Fine-Grained Visual Hybrid Deep Learning Model for Visually Challenged People (FGVHDL) invokes a parallel object detection stream and semantic refinement network to estimate the visual ambiguities. A smart glass frames are normalized and classified by YOLOv8 and YOLOv11 simultaneously. The result is a pair of complementary by Regions of Interest (RoIs). It provides a high recall for small/atypical obstacles by YOLOv11 and YOLOv8 for reliable and fast detector. The RoIs are used as inputs for a custom CNN with dilated convolutions to support multi-scale semantic classification. FGVHDL trained on a mixed dataset of 2,000 images featuring to classify the visual ambiguity in indoor residential scenes, urban streets and road configurations, and reached 93% of accuracy, 97.1% precision, 93.1% recall and 92.7% specificity with 50 ms processing time per frame (≈15 FPS).