Jul 2026· International Conference on Ubiquitous and Future Networks· pp. 1056-1061· 0 citations· 11 references
Abstract
Transfer learning with ImageNet-pretrained backbones is ubiquitous in computer vision, yet the question of which intermediate layer best serves a downstream task is typically resolved by convention rather than controlled measurement. Despite the prevalence of this practice, no systematic comparison of layer-wise frozen representations has been conducted across the modern architectures that dominate current pipelines. We present the Layer Transferability Profile, a reproducible evaluation protocol and an accompanying metric, the Layer-Task Transferability Score (LTTS), that together quantify how transferable information is distributed across depth. Applying the profile to three architectures, ResNet-50, EfficientNet-B0, and ViT-B/16, on three recognition benchmarks (CIFAR-10, CIFAR-100, and Food101), we find that transfer performance improves monotonically with depth and that the final or near-final representation is consistently optimal (LTTS 0-1.2%). This pattern contrasts with recent findings in speech and language foundation models, where intermediate layers can yield substantial gains, suggesting that the concentration of transferability in late layers is characteristic of supervised visual pretraining for recognition tasks rather than a universal property of deep representations. Notably, ViT-B/16 achieves the strongest transfer accuracy (78.1% on CIFAR-100, 94.6% on CIFAR-10, and 68.6% on Food-101) while using 768-dimensional features, outperforming ResNet-50’s 2048-dimensional and EfficientNet-B0’s 1280-dimensional representations. This indicates that representational quality per dimension varies substantially across architectures and can be more important than raw feature size when selecting a backbone for transfer.
Learning rate (LR) initialization and decay remain important factors in the optimization of deep vision networks. Although these models exhibit a clear hierarchical structure, training typically starts from a single global learning rate, with little explicit consideration of stage depth. This paper investigates a simple depth-aware alternative. We propose Asynchronous Learning Rate (ALR), which assigns depth-dependent initial learning rates to network modules according to their topological depth, and Smoothed Synchronous Decay (SSD), which coordinates the subsequent decay of heterogeneous parameter groups. A linear depth rule is adopted as a low-complexity, monotonic parameterization that is straightforward to implement across hierarchical architectures. The proposed strategy is evaluated on image classification and semantic segmentation benchmarks, including CIFAR-100, Mini-ImageNet, ImageNet-1K, Pascal VOC 2012, and LiTS. Additional ResNet-50 evaluations under benchmark-specific training protocols show that depth-scaled ALR improves Top-1 accuracy from 77.618% to 78.736% on native CIFAR-100 across five seeds and from 76.33% to 78.58% on ImageNet-1K. The results indicate that ALR generally outperforms uniform learning rate initialization, whereas SSD is most effective when combined with depth-aware initialization, particularly when layer-wise learning rates would otherwise follow misaligned decay trajectories. These findings suggest that a simple hierarchy-aware learning rate design can serve as an effective optimization refinement for hierarchical vision models.
Qiang He, Qiu Zong, Yi-Qi Wang et al.· IEEE Access· 0 citations
Vision foundation models are increasingly reused as frozen backbones for downstream visual recognition, making parameter-efficient adaptation a central problem. Prompt-based adaptation, including Visual Prompt Tuning (VPT), provides a lightweight way to specialize these models, but its layer-wise behavior remains poorly understood: performance is sensitive to prompt depth, placement, and task distribution, and gains on standard in-domain benchmarks do not always translate into robust generalization. We argue that this limitation is not solely an optimization issue, but a layer-wise information allocation issue: existing prompt-based methods lack principled control over what prompt-conditioned representations should preserve, suppress, and propagate across depth. Inspired by the Information Bottleneck principle, we introduce Prompted Information Bottlenecks (PIB), a framework that regularizes layer-wise compression-sufficiency trade-offs and promotes a more coherent cross-layer information path. The key idea is that effective adaptation should be minimal yet sufficient, retaining task-relevant local evidence in earlier layers while progressively discarding nuisance factors and redundant details in deeper layers. Extensive experiments show that PIB achieves strong performance across 34 datasets, reaching 92.1% on FGVC, 93.01% on HTA, and 77.33% on VTAB-1k, while tuning only 0.35% parameters on average across the main settings. Beyond benchmark accuracy, PIB helps explain the non-monotonic behavior of prompt capacity scaling, reduces shortcut reliance, and improves robustness under distribution shift and fine-grained recognition settings. These results position PIB as both a practical method and an information-allocation perspective for adapting frozen vision foundation models. Our code is available at https://github.com/itsnotacie/MM-26-PIB
Yuqi Li, Xi Xiao, Yunbei Zhang et al.· arXiv.org· 4 citations
Across cross-generator, post-processing, and in-the-wild benchmarks, PE-SPC surpasses the previous DINOv3 baseline and achieves new state-of-the-art results.
Wei-Han Cai, Hao Tan, Zichang Tan et al.· 0 citations
This work introduces \method, a Multi-scale Adaptive Vision Encoder, a Multi-scale Adaptive Vision Encoder that uses position-dependent gates to fuse shallow, intermediate, and deep features from a vision Transformer, preserving global semantics while enhancing edges, text, and local structure.
Pre-projector VDE tracks layer-wise accuracy and its top-ranked layers cover the oracle best layer on every task for the SigLIP-based LLaVA-Video, while giving region-level guidance for the CLIP-based Video-LLaVA.
Ruchen Liu, Yi Yang, Yiming Xu et al.· 0 citations
The Robust and Fine-grained training framework for CLIP-based vision-language models (RoFLIP) is proposed, enhancing both the robustness and granularity of vision-language alignment and underscore RoFLIP’s compositional reasoning and generalization abilities.
Yiwei Sun, Chuanbin Liu, Shancheng Fang et al.· International Journal of Com...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
