Does recovery advice help an agent finish a task after a tool fails? AgentFaultBench evaluates LangGraph and CrewAI agents performing specified tasks in an isolated GitLab installation. We inject tool errors and check both the resulting files and issues and the required reads, verification and final response. The promp...
A production-trace validation protocol, optional LLM-judge integration, and maintenance of expectations as workflows evolve are described, a practical pattern for expressing behavioral contracts within an existing Java release pipeline.
Agent tool approval and input validation do not, by themselves, bind a request to an approved source artifact. ToolContractGate provides a deterministic runtime contract evaluator for OpenAPI-derived agent tools. Each versioned ToolContract combines a stored OpenAPI fingerprint with access, schema, and behavioral decla...
This work presents VERITYGATE, a four-gate checker for declared evidence IDs, entities, numbers, and claim types, which checks a fixed schema; it does not verify every fact in the prose.
The authors' manual, post-hoc analysis is descriptive: in one inspected case, optical character recognition (OCR) and block grouping merged the relevant passage into another block, and the system answered from unrelated text.
Finding a relevant paper is not the same as producing a verifiable answer from it. LitTraceQA requires canonical paper identifiers, exact evidence at the page or object level, and typed answers that match the evaluator. We call the separation between source access and scorer-visible correctness the grounding contract g...
Sachin Gupta, Divya Godara· Proceedings of the 1st Works...· 1 citation· ⚡1
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.