Single Domain Generalization Benchmark: Current State and Challenges

BPG: Balancing Plasticity and Generalization for Domain Incremental Learning

Deep neural networks excel in various tasks but struggle to generalize across evolving data distributions, leading to significant performance degradation under domain shifts. Domain incremental learning (DIL) addresses this challenge by enabling models to continuously adapt while retaining prior knowledge. Among existing DIL approaches, the parameter-isolation paradigm achieves state-of-the-art performance. However, these methods often adopt a one-size-fits-all approach to adapt to new domains, resulting in either insufficient learning capacity or redundant parameters. In this work, we propose BPG, a unified framework that addresses both challenges through two complementary components: BPG-Adapter, which dynamically determines each domain's adapter hidden dimension based on domain-specific feature separability, and BPG-Inference, a soft domain mixture strategy that integrates multiple domain-specific models at test time, mitigating domain ID misselection. Experimental results on DomainNet, CDDB, and CORe50 demonstrate that BPG consistently outperforms uniform adapter-based approaches and hard domain selection strategies, achieving state-of-the-art average accuracy while reducing forgetting to as low as 0.22% on DomainNet.

Deep Learning Enhancements Using Pretraining and Fine-Tuning

Experimental findings show that pretrained models outperform those trained from scratch in terms of accuracy, convergence speed, and robustness, and a unified framework is proposed to integrate both processes in a deep learning pipeline.

Multimodal Domain Generalization via Adaptive Dual-Objective Feature Learning

Benchmarking Pre-Trained Feature Extractors: A Comparative Study Across Deep Learning Tasks

The rapid growth of artificial intelligence and deep learning has revolutionized diverse domains, including computer vision where transfer learning through pre-trained models has become a fundamental technique for enhancing performance of models and reducing computational costs. Although numerous pre-trained deep learning models have been widely adopted, their effectiveness varies significantly across application domains and datasets. The lack of comprehensive comparative studies makes it challenging for researchers to identify the most suitable models for achieving optimal performance in specific tasks. This study systematically compares seven pre-trained feature extractors across three architectural families, convolutional neural networks (CNNs), Vision Transformers (ViTs), and self-supervised models to provide practical guidance on model selection for downstream deep learning tasks. These models were evaluated on five benchmark datasets. Features were extracted with frozen weights and evaluated using linear probing, k-nearest neighbor retrieval, and 5-shot classification. ConvNeXt-B achieved the highest mean linear probe accuracy (90.40%), while DINOv2-S produced the best feature geometry for retrieval tasks (87.64%). CLIP-ViT-B/32 demonstrated the strongest cross-domain transfer, leading on texture recognition and satellite imagery few-shot classification. Older CNN architectures lagged significantly behind modern models by approximately 7%. Overall, this study simultaneously evaluates multiple feature extractors across multiple visual domains and evaluation protocols.

Uncertainty Generation Meta-training for Cross-domain Few-shot Learning

Cross-domain few-shot learning focuses on transferring knowledge from a source domain to an unseen target domain, but traditional methods often rely on simple additive perturbations, which limit their effectiveness in handling domain shifts. To address this limitation, we propose an innovative meta-learning framework augmented by uncertainty generation. Our approach introduces a novel gradient-based uncertainty generation method that treats feature statistics, such as mean and variance, as probabilistic representations assuming an unknown distribution. These statistics are manipulated through task gradients, which allow the model to capture the range of possible variations, thereby generating uncertainty feature distributions that simulate domain biases. Additionally, we incorporate a causal invariant information (CII) module to extract causal features and improve the consistency of task gradients across domains. This module ensures that the task gradients reflect domain-invariant features, reducing the impact of domain-specific noise and providing a more reliable framework for constructing uncertainty boundaries. Together, these components enable the development of an asymptotic meta-learning optimization algorithm that learns generalized knowledge from uncertainty features while optimizing the uncertainty distribution boundaries. Experimental results across nine datasets demonstrate that our method improves the current state-of-the-art by an average of 3.11%, with key innovations in using probabilistic feature statistics, gradient-based uncertainty modeling, and causal feature extraction that enhance cross-domain knowledge transfer more effectively than existing techniques.

Efficient unsupervised domain adaptation via self-supervised vision transformer and synergistic cross-domain alignment

Unsupervised domain adaptation (UDA) aims to mitigate domain shift, where the distribution of labeled source data differs from that of unlabeled target data. Despite recent advances, existing methods often rely on fine-tuning large backbone models, which leads to high computational cost and limits scalability in resource-constrained environments. This limitation highlights the need for parameter-efficient approaches that maintain strong performance with reduced training complexity. Self-supervised foundation models such as DINOv2 provide highly transferable representations and raise the question of whether effective domain adaptation can be achieved without full fine-tuning. To address this question, we propose Efficient Unsupervised Domain Adaptation (EUDA), a parameter-efficient framework that leverages a frozen DINOv2 backbone as a feature extractor and updates only a lightweight bottleneck and classification head. We also adopt a synergistic domain alignment loss (SDAL), which combines cross-entropy (CE) and maximum mean discrepancy (MMD) to promote both discriminative learning and cross-domain alignment. Experimental results on Office-Home, Office-31, VisDA-2017, and DomainNet demonstrate that EUDA achieves competitive performance across diverse domain complexities, while reducing the number of trainable parameters by 42 to 99.7%. These results show the suitability of the proposed method for resource-constrained and distributed environments.