Conformal prediction for multi-label learning: a review of methods and guarantees.

Abstract

Multi-label learning (MLL) is a machine learning paradigm that aims to predict a set of labels for each instance, rather than a single class. Such tasks arise in a wide range of real-world applications and pose significant challenges, including an exponentially large output space, dependence among labels and often severe label imbalance. These challenges amplify predictive uncertainty, making reliable uncertainty quantification essential. Conformal prediction (CP) is an attractive answer: it converts model outputs into prediction regions with distribution-free, finite-sample guarantees under the sole assumption of data exchangeability. Several adaptations of CP to the multi-label setting have been proposed. Yet these vary widely in scoring constructions, output types and targeted guarantees. This review consolidates the landscape of CP adaptations for MLL. It places existing approaches under a unified framework, examining the types of outputs and guarantees they provide, where label dependencies are incorporated, and how inference cost scales with the number of labels. It provides an in-depth analysis of all approaches using common notation, identifying their key characteristics along with their practical implications and assessing their strengths and limitations. Finally, it compares approaches side-by-side, highlighting trade-offs among guarantee types, precision of regions, compactness of outputs and scalability. This article is part of the theme issue 'Advancing uncertainty quantification in AI systems'.