A Multilevel Visual and Textual Framework for Near-Duplicate Diagram Detection in Electronic Documents

Abstract

Near-duplicate diagram detection in electronic documents is challenging because diagram identity depends on graphical structure, spatial composition, and textual labels, while reused images may undergo compression, cropping, rotation, photometric changes, or perspective distortion. This study proposes a cascaded multimodal framework combining perceptual hashing, Siamese Residual Network with 18 layers (Siamese ResNet18), Distillation with No Labels, ver. 2 (DINOv2)visual representations, and a text-similarity classifier. A controlled benchmark was constructed from Artificial Intelligence 2D Diagram Dataset (AI2D) using Light, Medium, and Hard transformations, with source-grouped splitting by base_id to prevent leakage across training, validation, and test sets. Perceptual hashing achieved Area Under the Receiver Operating Characteristic Curve (ROC-AUC) = 0.7483, while Siamese ResNet18 increased ROC-AUC to 0.8599. DINOv2 provided the strongest visual performance, achieving Accuracy = 0.9933, F1-score = 0.9933, ROC-AUC = 0.9992, and Average Precision = 0.9994; F1-score remained 0.9901 for Hard transformations. Visual fusion increased ROC-AUC to 0.9995, and full multimodal fusion reached ROC-AUC = 0.9998. At an early-exit threshold of 0.95, 27.6% of pairs were resolved at the hashing level. These results support the coarse-to-fine design on the constructed AI2D-derived benchmark. A targeted hard-negative stress test revealed substantially higher false-positive rates under deliberately matched spatial layouts, with an overall False Positive Rate (FPR) of 0.48 for DINOv2 and 0.16 for full multimodal fusion. Generalization to naturally reused or redrawn diagrams, larger and more diverse hard-negative collections, and Optical Character Recognition (OCR)-derived text remains to be evaluated.