Skip to content

How Synthetic Labels Improve Conformal Prediction: A Perspective on Conditional Coverage

Sep 2026 · arXiv.org · Vol abs/2609.33482 · 0 citations · 30 references
Computer Science Mathematics

TL;DR

This work introduces prediction-powered quantile learning: a synthetic-labeled pool estimates pinball risk, paired trusted and synthetic outcomes correct its bias, and an independent trusted split performs final conformalization.

Abstract

Conformal prediction provides distribution-free finite-sample marginal coverage, but post-hoc calibration data may be too scarce to learn how uncertainty varies across inputs. Meanwhile, abundant covariates can often be labeled cheaply by domain models or general-purpose language models. We study whether these synthetic labels can improve conditional coverage when only a small trusted sample is available. Building on score-quantile regression, we introduce prediction-powered quantile learning: a synthetic-labeled pool estimates pinball risk, paired trusted and synthetic outcomes correct its bias, and an independent trusted split performs final conformalization. Profiling pinball risk over scalar corrections reveals that population conditional-coverage error is its functional gradient; the corresponding Hessian removes global shifts and weights remaining shape error by boundary density. Composing this geometry with prediction-powered learning yields a three-resource expansion and a benefit--cost rule for synthetic power. Across eight regression benchmarks, synthetic-powered quantile learning substantially improves downstream conditional coverage while preserving marginal validity and producing more compact prediction sets. A human-rating study finds similar gains from external LLM labels and exposes a quality--quantity--cost tradeoff.

View source

Similar papers

Sharp training-conditional coverage for conformal prediction under covariate shift

Weighted split conformal prediction reweights calibration scores by the likelihood ratio between the test and training covariate distributions and guarantees marginal coverage under covariate shift. We study its coverage conditional on the calibration data. An elementary argument, based on a single concentration inequa...

Mehrdad Pournaderi · 1 citation

Conformal Prediction under Exponential-Tilt Joint Shift

This work uses Exponential Tilt Reweighting Alignment (ExTRA), introduced for classification by Maity et al. (2023), to estimate structured distribution shifts and compares using its estimated weights in conformal calibration with additionally tilting the source predictive distribution.

Seung-Jin Choi · 1 citation
Preprint Aug 2026

Local conformal prediction for individual causal effects

Standard CATE estimators become inadequate under strong treatment-effect heterogeneity: confidence intervals for conditional means need not cover individual counterfactual effects. We propose an Individualized Causal Prediction (ICP) framework that constructs finite-sample valid conformal prediction intervals for the i...

Fernando Delbianco, Fernando Tohmé · 0 citations

Bounded-load calibration splits for differentially private graph conformal prediction

It is shown that private graph calibration can be framed as a structural problem: choosing a calibration set that is large enough to provide useful calibration, yet sparse enough that the privacy noise remains manageable, and it is formalized that it is NP-hard.

Patrick Indri · 0 citations

Scaling Full Conformal Image Classifiers

Targeted Full Conformal Prediction (T-FCP) is introduced, which uses a lightweight inductive conformal predictor to prune unlikely labels and applies FCP only to the remaining candidates, reducing computation while retaining the formal guarantee of the combined conformal procedure.

Julio Silva-Rodríguez, E. Konukoglu · 0 citations
#machine learning Preprint Sep 2026

Split Conformal Prediction with Label-Shift-Adjusted Bayesian Scores

The LSA score is proposed, a nonconformity score derived from a posterior predictive tilting identity that shows that the target predictive is an importance-weighted transformation of the source predictive, which is used to derive a direct correction to the Bayesian score.

Hyeonsu Lee, Juyeong Kim, Erkhembayar Jadamba et al. · 3 citations

Related blog posts

MIT News · Artificial Intelligence Nov 17, 2026

Learning the language of machine learning

By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.

Microsoft Research Blog Nov 16, 2026

TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient

AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.

MIT News · Artificial Intelligence Oct 23, 2026

Q&A: How AI came to model the “messy” human mind

Assistant Professor Benjamin Lindquist studies how engineers deliberately injected randomness into early AI, giving rise to deep-learning models that sometimes hallucinate.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.