Uncertainty-Guided Adaptive Knowledge Distillation for Lightweight Cross-Domain Object Detection
TL;DR
This article introduces Uncertainty-Guided Adaptive Knowledge Distillation (UGAKD), a novel framework designed to enhance UDA while simultaneously reducing model size through targeted knowledge distillation and proposing a two-stage, difficulty-aware training scheme to facilitate learning.
Abstract
Unsupervised Domain Adaptation (UDA) is essential for adapting object detection systems to diverse operational environments without requiring domain-specific labeled data. In parallel, real-time performance is crucial for deploying these detectors in intelligent vehicles. This article introduces Uncertainty-Guided Adaptive Knowledge Distillation (UGAKD), a novel framework designed to enhance UDA while simultaneously reducing model size through targeted knowledge distillation. Given that adversarial learning is a common approach in UDA, often utilizing a domain classifier to identify domain-invariant features, UGAKD leverages the localization of these domain-invariant features to guide the distillation process. Furthermore, we propose a two-stage, difficulty-aware training scheme to facilitate learning, which emphasizes domain-invariant features to boost distillation efficacy. Experimental results across several challenging scenarios, including transitions from synthetic to real-world environments, varying weather conditions, and shifts between real and stylized domains, demonstrate that UGAKD effectively reduces model complexity while improving detection accuracy. Specifically, UGAKD decreases the number of parameters by over 37% and FLOPs by over 47%. Compared with the baseline fine-grained feature imitation method, UGAKD achieves an mAP improvement of 1.0% to 1.7%, highlighting its effectiveness in maintaining robust object detection across diverse settings and its suitability for applications that require both efficiency and adaptability.