Seq@K: Evaluating Language Models Under Sequential Feedback
This work introduces Seq@K, a natural evaluation protocol that measures whether a model can solve a problem within K sequential attempts, provided feedback after each previous failure, and establishes how Seq@K offers a more faithful metric for intelligent systems that typically operate in such feedback loop, and the n...
Thomson Yen, Gilbert Yang, Nai-Meng Ye et al.
· 0 citations