What Does Disagreement Mean? A Semantic Identifiability Frontier for Pairwise Preference Learning
TL;DR
A nonparametric latent semantic model with three querywise quantities, which proves an exact semantic fiber theorem and derives finite-sample estimators, nonasymptotic concentration bounds, and semantics-aware action-recovery guarantees, and validate the theory on theorem-matched synthetic data.
Abstract
Pairwise comparison data are a standard supervision signal in preference learning, but disagreement in such data can reflect stable latent heterogeneity, ephemeral stochasticity, or non-binary responses hidden by a forced-choice interface. We study which of these semantic distinctions are identifiable from the feedback channel itself. We introduce a nonparametric latent semantic model with three querywise quantities: aggregate directional preference $\mu_q$, persistent disagreement $\tau_q$, and non-binary mass $\alpha_q$. Under the anonymous binary channel, we prove an exact semantic fiber theorem: $\mu_q$ is point-identified, whereas $(\tau_q, \alpha_q)$ vary over an explicit feasible region compatible with the same binary law. Same-annotator repeats identify $\tau_q$ through a covariance identity, abstention-enabled feedback identifies $\alpha_q$ directly, and together they point-identify the full semantic triple. We also derive finite-sample estimators, nonasymptotic concentration bounds, and semantics-aware action-recovery guarantees, and validate the theory on theorem-matched synthetic data.
