Jul 2026
How Far Are We from Detecting Flaky Tests? On the Limits of Code-Based Detection
It is found that widely used benchmarks contain shortcuts that inflate reported F1 scores and that evaluation protocols overstate generalizability, and reframe flakiness prediction around whether an observed failure is flaky and how likely a test is to fail given its execution environment.
Ömer Oktay Gültekin, Alexander Berndt, Jonathan Bell et al.
· arXiv.org · 0 citations