TCSAlgBench provides a refreshable testbed for measuring progress in model reasoning and studying how agent workflows support research-level proof discovery, and evaluates ten model configurations from four families under direct inference and prover-verifier discussion, and compares four agent workflows under matched m...
Chu-Tong Yang, Xi-Yuan Zhang, Yu Huang et al.· 0 citations
SA-Pass (*Semantic Alignment Pass*), which tests formal statements using auxiliary statements called shadows that characterize the intended statement, is proposed, which achieves binary agreement with expert judgments.
Hojae Han, Jongyoon Kim, Sanghyeok Park et al.· 2 citations
Vero is introduced, the first benchmark to evaluate joint implementation and proof synthesis at the repository level and an audit mechanism where agents are allowed to formally prove unsatisfiability of provided specification or incorrectness of reference code, which surfaces and corrects latent code and specification...
Zhe Ye, Hantao Lou, Yuechun Sun et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.