Influence of Image Quantization on the Sim-to-Real Gap of Synthetically Trained Deep Learning Models

Abstract

For domains where data is scarce, costly, or privacy-sensitive, synthetic data generation offers a reasonable alternative as a basis for the training of deep learning models. Yet, a persistent challenge arises from the sim-to-real gap, which can deteriorate a synthetically trained model's performance when applied to real-world data. While many works following compensation strategies like domain randomization explored potential factors of influence, the influence of the image quantization level has not been considered yet. This work examines the potential of explicit JPEG compression of synthetic images, investigating the effect of different levels of image quantization. The presented results show that the utilization of JPEG compression acts as a regularization measure for the training of object detection models, forcing the models to broader generalization and helping bridge the sim-to-real gap during inference.