SWE-Test: Benchmarking LLM Vulnerability Discovery via Input Prediction
Vulnerability discovery is becoming an important ability of large language model (LLM) agents: agents that silently miss real defects leave critical software exposed. Rigorously measuring this ability is therefore urgent, but existing benchmarks are gameable through data contamination, score recall against an unknowabl...
Yuan-Xiang Shi, Jia-Yi Lin, Xuan-Yong Lin et al.
· 0 citations