A Multi-Level Synthetic–Real Data Fusion Framework for Underwater Object Detection

Abstract

Underwater Object Detection (UOD) is essential for marine monitoring, ecological assessments, and autonomous underwater operations, yet progress is significantly hindered by the scarcity, cost, and complexity of collecting and annotating real-world underwater imagery. Synthetic datasets generated from high-fidelity 3D environments offer a promising alternative, but the domain gap caused by unrealistic visual conditions and incomplete feature distributions limits their applicability. This study systematically evaluates multi-source training strategies at the data, pixel, object, and training levels, comparing two deep learning architectures, Faster R-CNN and YOLOv11, and evaluating performance using mean Average Precision (mAP) across five-fold cross-validation and an external test set. Results show that synthetic data is most beneficial in data-scarce settings, preventing model collapse in two-stage detectors (TSD) and improving overall mAP by up to 35.4% in single-shot detectors (SSD). When integrating synthetic data into the full real-world dataset, realism-enhancing pre-processing is most beneficial for SSD, improving performance by 8.0% over the real-data baseline, whereas hybrid composition provides larger gains for TSD (9.9% over the baseline), highlighting the importance of object-level realism and spatial consistency in region-based detectors. Curriculum learning yields more modest but consistent improvements for both detectors (TSD: 2.7%; SSD: 4.7%). Overall, the findings demonstrate that synthetic data effectiveness is not inherent but conditional on how it is generated, structured, and integrated relative to the target architecture and available real data, providing practical guidance for scaling UOD in marine monitoring applications.