Skip to content
Preprint

Ranking Experiments under Sequential Sampling

Aug 2026 · 0 citations
Economics

Abstract

We compare statistical experiments when observations are inexpensive and can be acquired sequentially until the decision maker chooses to stop. We introduce two orders. Small-cost decision dominance asks which of two equally priced experiments is eventually preferred in every decision problem as the per-observation cost vanishes; large-budget stopping dominance asks which experiment can reproduce every terminal experiment attainable from the other under all sufficiently large expected-sample budgets. Our main result shows that, for generic pairs, the two orders coincide and are both characterized by strict dominance of every pairwise Kullback--Leibler divergence. The key step is a uniform exact-conversion theorem: any finite-output stopping policy based on one experiment can be reproduced exactly using another, with first-order expected-sample requirements determined by pairwise KL rates and a square-root remainder that is uniform over policies.

View source

Similar papers

Preprint Nov 2026

Adaptive pairwise preference elicitation for decision support: identifying the best alternative with fewer judgments

Interactive decision support systems often require repeated preference judgments, although identifying the best alternative may not justify eliciting all \(n(n-1)/2\) pairwise comparisons. We propose ALEC (Adaptive Lexicographic Elicitation of Comparisons), which re-ranks the alternatives after each judgment and asks t...

L. A. Calvo, Jiri Mazurek · 0 citations
Preprint Oct 2026

A Simple and General Approach to top-$k$ Selection

This paper considers the Bayesian top-$k$ identification problem, wherein a decision maker is tasked with selecting $k$ items from a pool of items of unknown quality. To assess the items, the decision maker adaptively allocates a fixed sampling budget, where each sample of an item yields a noisy observation of its qual...

M. Goldberger, N. Rudi · 0 citations
#machine learning Preprint Oct 2026

Learning Distributions from Samples and Qualities

We study parameter estimation when each sample from a continuous parametric distribution is accompanied by a binary quality label governed by the same unknown parameter. By imposing hard feasibility constraints, deterministic labels can accelerate estimation from the classical $n^{-1/2}$ rate of smooth quality-aware M-...

Marc Jourdan, Nicolas Flammarion · 0 citations
#machine learning Preprint Oct 2026

How Many Repeated Pairwise Comparisons Are Needed for Ranking under Heterogeneity?

We study ranking models by population-average utility from pairwise comparisons when preferences vary across users and tasks. Prior work shows that a single comparison per user can be insufficient to identify the alternative with the highest average utility, even with arbitrarily many users (Golz et al., 2025). We inve...

Shashaank Aiyer, Han Shao · 0 citations
#machine learning Preprint Oct 2026

Metropolis-Hastings Dominates Importance Resampling for Policy Composition

Post-training a large language model (LLM) often requires exploring trade-offs between multiple rewards, but retraining for each trade-off is expensive. Decoding-time policy composition allows these trade-offs to be adjusted by combining reward-specific policies at inference time. This composition targets a weighted pr...

Alexey Kurennoy, R. Yarullin, Fergal Reid · 0 citations

The cost of useful natural gradient updates

What information is needed to turn a natural-gradient direction into a useful finite update? Under a population Kullback-Leibler (KL) budget, we call a step useful if it is feasible and loses at most a fraction $\varepsilon$ of the best feasible gain along the direction. We construct a four-state exponential family who...

Subhransu S. Bhattacharjee, Dylan Campbell, Rahul Shome · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.