Adversarial training, one of the most effective methods for enhancing neural network robustness, is typically formulated as a min-max game between an attacker and a defender. Despite its success, most adversarial training methods suffer from robust overfitting, leading to a significant gap in robustness between the training and test sets. In this paper, we theoretically analyze this phenomenon through the lens of feature learning. Our analysis reveals that fitting non-robust features is the root cause of overfitting, which is exacerbated by the attacker and defender with unbalanced optimization strength. Specifically, a defender that is too strong forces the model to learn non-robust features, while an attacker that is too strong hinders the feature learning process and leads to extra computational costs. To address this issue, we propose Balanced Instance Adaptive Defense (BIAD), which dynamically balances attackers and defenders at the instance level. BIAD classifies training examples into robust and non-robust groups and strengthens the attacker and regularization for robust examples while weakening them for non-robust ones. Experimental results across various architectures and datasets (CIFAR-10, CIFAR-100, Tiny-ImageNet, and ImageNet) demonstrate that BIAD achieves higher training efficiency, better accuracy-robustness trade-offs, and effectively mitigates robust overfitting.
Xin-Yue Zhang, Shaocong Wu, Qiben Shan et al.· IEEE Transactions on Image P...· 0 citations
CLIP, as a vision-language model, has significantly advanced Open-Vocabulary Semantic Segmentation (OVSS) with its zero-shot generalization. Despite its success, its application to OVSS is limited due to its initial image-level alignment training, which affects its performance in tasks requiring detailed local context. Our study delves into the impact of CLIP's [CLS] token on patch feature correlations, revealing a dominance of the "global patch" that hinders local feature discrimination and demonstrating the reasons, trends and impacts of its occurrence. To overcome this, we propose CLIPtrase, a novel training-free semantic segmentation strategy, which enhances local cognition through recalibrated self-correlation among patches and quadrant mask ensemble strategy. This approach demonstrates significant improvements in segmentation accuracy and the ability to maintain semantic coherence between objects. The experiments indicate a 22.8% average improvement over CLIP in 9 segmentation benchmarks, surpassing current state-of-the-art training-free methods. Moreover, the proposed method can be seamlessly adapted to other tasks such as unsupervised semantic segmentation and multi-label classification with satisfactory performance, which demonstrate the generalization of our approach. The code is available at https://github.com/leaves162/CLIPtrase.
Tong Shao, Zhuo-Tao Tian, Yun-Yang Mo et al.· IEEE Transactions on Pattern...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
