From Answers to Audit Opinions: Cost-Aware Expert Verification of Financial Artifacts from Tool-Using LLM Agents

Abstract

Tool-using LLM agents increasingly produce financial artifacts such as ratio tables, evidence-backed memos, and spreadsheet-style analyses. Existing evaluations often reduce such outputs to binary correctness or faithfulness labels, overlooking whether the artifact has sufficient evidence, whether errors are material or pervasive, and whether evidence gaps should prevent acceptance. We adapt audit-opinion logic to agentic artifact verification as a research-label framework with four labels: Clean, Qualified, Adverse, and Disclaimer. The proposed audit certificate turns each artifact into a checkable workpaper by linking claims to provenance, formulas, verification outcomes, scope limits, and Critical Artifact Matters. Our evaluation package combines SEC-backed cases, oracle-backed injected failures, expert adjudication, blind re-audit, hard-case stress diagnostics, and an expanded 400-case oracle-backed checker-field diagnostic. In the expanded diagnostic, full-certificate checker fields remove false-clean labels that remain common under artifact-only and partial-certificate views. The completed Phase 3 expert batch shows that oracle-backed perturbation labels and expert audit opinions diverge in materiality-sensitive cases, while blind re-audit shows substantial within-expert stability.