Structure-Grounded Document Agents for Faithful Long-Context Reasoning

Abstract

Large language models (LLMs) struggle with longdocument reasoning: naively packing entire documents into the context window leads to degraded performance, while retrievalaugmented generation (RAG) fragments documents into isolated chunks that lose structural coherence. Moreover, even when existing agents produce correct answers, their intermediate reasoning steps are often unfaithful to the retrieved evidence. We propose StructAgent, a document-oriented LLM agent that integrates structure-aware navigation, sequential reading, and evidence-constrained generation into a unified agentic loop. StructAgent first parses the document hierarchy (sections, tables, and cross-references), then navigates this structure tree to locate relevant regions, reads contiguous passages to preserve local context, and finally generates answers whose reasoning chains are explicitly grounded in cited evidence. We evaluate StructAgent on three long-document benchmarks-QASPER, HotpotQA, and QuALITY-measuring both answer accuracy (F1) and reasoning faithfulness via two newly introduced metrics: Evidence-Reasoning Consistency (ERC) and Reasoning-Answer Consistency (RAC). Experimental results show that StructAgent achieves absolute F1 gains of 3.1-10.9 points depending on the baseline and dataset, along with higher faithfulness scores compared to vanilla RAG, long-context LLMs, and ReAct-based agents, especially on the length-controlled QASPER setting.