Findings show that functional-only evaluation overestimates agents'ability to satisfy the full requirements of repository-level repair tasks, and introduces SWE-Gate, a repository-level benchmark for software engineering agents that explicitly evaluates review constraint compliance alongside functional correctness.
Xin He, Yan-Lin Wang, Ming-Wei Liu et al.· 0 citations
PTA-IRT is proposed, a Privileged Trajectory-Aware Item Response Theory framework that fuses process and outcome signals and consistently outperforms prior IRT baselines on score and ranking recovery across four SWE benchmarks.
Ke-Feng Duan, De-Wu Zheng, Yan-Lin Wang et al.· 0 citations
ACToR identifies critical tokens during generation and triggers targeted retrieval on demand to provide repository context at these decisive positions, and designs a position-aware weighting method for dense retrievers to prioritize context that is more informative for generation.
Ke-Feng Duan, De-Wu Zheng, Yan-Lin Wang et al.· 0 citations
This work proposes WebDesignIter, a framework built around a persistent knowledge graph (WebAppArchKG) that fuses repository structure with design knowledge and keeps both in sync across development cycles, and outperforms every general-purpose coding agent Claude Code, OpenHands, SWE-Agent, Codex CLI on every model co...