Weakly-Supervised Learning with Partial Multi-Labels by Leveraging Dual Label Correlation Perspectives

Abstract

Multi-Label Learning (MLL) refers to inducing multi-label prediction models from the precisely labeled training dataset. However, in many real-world scenarios, e.g., crowdsourcing annotations, the training datasets are often only partially valid, where each training instance is associated with a candidate label set, covering ground-truth labels but also with irrelevant ones. Naturally, learning with such datasets, formally referred to as Partial MLL (PML), involves many noisy supervised signals, hence imposing a significant challenge to the prediction model induction. To meet this challenge, we purify the noisy supervised signals by formulating the latent label distribution, i.e., the probability of a candidate label being a ground-truth one, and then jointly learn it with the prediction model by minimizing their regularized Wasserstein distance, i.e., a robust distance for distributions as well as involving label correlations. Therefore, we propose a novel PML method, namely Wasserstein PML with dual Label Correlation Perspectives (Wpml3cp), solved by gradient descent with an augmented Lagrange multiplier technique. To further enhance the robustness of Wpml3cp against exceptionally high ratios of irrelevant labels, we extend it with a Dual-branch Competitive Cleansing mechanism, leading to Wpml3cp-D. Besides, we also analyze the generalization error bound and time complexity of Wpml3cp and Wpml3cp-D. The extensive experiments are constructed by comparing Wpml3cp and Wpml3cp-D with existing PML baselines across synthetic and real-world datasets, and empirical results demonstrate that Wpml3cp and Wpml3cp-D can outperform the PML baselines in various noisy levels.