PetriBench is introduced, a compact, fully self-contained, and scalable benchmark for evaluating LLM reasoning over dynamic state spaces using Petri nets, a mature formalism for modeling real-world concurrent and distributed systems.
Pyrros Koussios, Benjamin Jäger, John Hua Yao et al.· 0 citations
HarnessCompass is proposed, a novel automatic harness evolution framework built around constrained evolution, proactive feedback, and component-wise optimization that improves Pass@1 from 54\% to 66\% in only 5 evolution iterations, outperforming AHE in both effectiveness and evolution efficiency.
Luan Zhang, Ruo-Chen Zhou, Dan-Dan Song et al.· 10 citations· ⚡1
This work proposes MedQPA, an automatic and generalizable report evaluation technique that uses question proposing and answering to enable controllable, structured reasoning grounded in medical domain knowledge and the factual correctness of the report.
Weijie Liang, Xiyue Zhu, Ruike Zhu et al.· Annual Meeting of the Associ...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.