Same Winners, Different Success Rates: Evaluating How LLM Agents Recover from Failures
It is proved that certifying exact population ties is impossible in finite time, and that the assignment of outcomes to checkpoints carries information beyond marginal outcome distributions, and proposed four diagnostic quantities that expose this failure mode with no additional data collection.