Diego Mesquita

Few Trials, Many Metrics: Beta Mixtures for Low-Budget AI Evaluation

Evaluating a stochastic AI system often requires many repeated samples per instance, especially for metrics defined over long horizons. Recent work has shown that modeling instance-level success probabilities with a Beta distribution recovers Pass@$k$ from far fewer samples than direct estimation. We show that this fra...