Multi-Label Learning (MLL) refers to inducing multi-label prediction models from the precisely labeled training dataset. However, in many real-world scenarios, e.g., crowdsourcing annotations, the training datasets are often only partially valid, where each training instance is associated with a candidate label set, covering ground-truth labels but also with irrelevant ones. Naturally, learning with such datasets, formally referred to as Partial Multi-label Learning (PML), involves many noisy supervised signals, hence imposing a significant challenge to the prediction model induction. To meet this challenge, we purify the noisy supervised signals by formulating the latent label distribution, i.e., the probability of a candidate label being a ground-truth one, and then jointly learn it with the prediction model by minimizing their regularized Wasserstein distance, i.e., a robust distance for distributions as well as involving label correlations. Therefore, we propose a novel PML method, namely Wasserstein Partial Multi-Label Learning with dual Label Correlation Perspectives (Wpml3cp), solved by the gradient descent with an augmented Lagrange multiplier technique. To further enhance the robustness of Wpml3cp against exceptionally high ratios of irrelevant labels, we extend it with a Dual-branch Competitive Cleansing mechanism, leading to Wpml3cp-D. Besides, we also analyze the generalization error bound and time complexity of Wpml3cp and Wpml3cp-D. The extensive experiments are constructed by comparing Wpml3cp and Wpml3cp-D with existing PML baselines across synthetic and real-world datasets, and empirical results demonstrate that Wpml3cp and Wpml3cp-D can outperform the PML baselines in various noisy levels.
Ximing Li, Yuanchao Dai, Bing Wang et al.· ACM Transactions on Knowledg...· 0 citations
Positive-Unlabeled (PU) learning refers to a specific weakly-supervised learning paradigm that induces a binary classifier with a few positive labeled instances and massive unlabeled instances. To handle this task, the community has proposed dozens of PU learning methods with various techniques, demonstrating strong potential. In this paper, we conduct a comprehensive study to investigate the basic characteristics of current PU learning methods. We organize them into two fundamental families of PU learning, including disambiguation-free empirical risks , which approximate the expected risk of supervised learning, and pseudo-labeling methods , which estimate pseudo-labels for unlabeled instances. First, we make an empirical analysis on disambiguation-free empirical risks such as uPU, nnPU, and DistPU, and suggest a novel risk-consistent set-aware empirical risk from the perspective of aggregate supervision. Second, we make an empirical analysis of pseudo-labeling methods to evaluate the potential of pseudo-label estimation techniques and widely applied generic tricks in PU learning. Finally, based on those empirical findings, we propose a general framework of PU learning by integrating the set-aware empirical risk with pseudo-labeling. Compared with existing PU learning methods, the proposed framework can be a practical benchmark in PU learning.
Yuanchao Dai, Zhengzhang Hou, C. Li et al.· Neural Information Processin...· 1 citation