CANNE: CLIP-Based ANNE Selection for Noisy-Label Learning

Abstract

Learning with noisy labels (LNL) remains challenging, especially when the identification of clean samples relies heavily on the predictions of the model being trained. In such cases, early-stage selection errors may be reinforced during iterative optimization, leading to unreliable supervision. To alleviate this issue, a two-stage framework, termed CANNE, is proposed by combining Contrastive Language–Image Pre-training (CLIP)-based conservative offline cleaning with Adaptive Nearest Neighbors and eigenvector-based sample selection (ANNE)-based online refinement. Specifically, a high-confidence clean seed set is first constructed using two complementary probability sources derived from frozen CLIP representations and reliability criteria, including class-wise loss modeling and prediction consistency. This seed set is then used as a set of reliable anchors during the subsequent ANNE training process, where online feature- and neighborhood-based refinement further recovers and adjusts sample partitions. In this way, CANNE uses external vision–language priors to provide conservative and persistent guidance while preserving the adaptive recovery ability of online noisy-label learning. Experimental results on CIFAR-10, CIFAR-100, Animal-10N, and Mini-WebVision, together with additional evaluation under open-set noise, show that the proposed method achieves competitive performance across diverse noisy-label settings. In particular, CANNE achieves 96.6% and 96.3% best accuracies on CIFAR-10 under 80% and 90% symmetric noise, respectively, and 81.0% and 79.0% on CIFAR-100 under 20% and 50% symmetric noise. Additional repeated-run, threshold-sensitivity, and runtime analyses further indicate that the CLIP-based seed set provides stable guidance with only moderate computational overhead.