A Hybrid Sim-to-Real Framework for Industrial Object Detection Using Synthetic Data Augmentation and Inference-Time Fusion

Abstract

We present a hybrid sim-to-real framework for the accurate detection of delivery boxes in a warehouse environment, without depending on large-scale, annotated real-world data. The framework combines realistic synthetic data generation using NVIDIA Isaac Sim to supplement limited real-world annotations, and an inference-time Weighted Box Fusion (WBF) strategy to mitigate distribution mismatch during inference between synthetically generated and real data. We systematically scale synthetic data augmentation across varying dataset proportions while maintaining strict train/test separation and evaluating models only on real images to prevent bias. Our experimental results using YOLO26 as an object detection model demonstrate that while performance gains saturate beyond 50% synthetic augmentation for the standalone model, inference-time fusion consistently delivers superior results across all tested configurations, even when the standalone YOLO26 model using the synthetic data plateaus. The fused approach achieves robust detection metrics, achieving 95% mAP@50, 95% precision, and 90% recall.