ABSP: Attention-Based Semantic Preservation for Faithful Context Compression

Abstract

Retrieval-augmented generation (RAG) reduces hallucination in large language models by conditioning generation on retrieved evidence, but retrieved contexts are often compressed before inference to reduce cost and latency. Prior work evaluates compression primarily through answer accuracy, leaving open whether different compression strategies affect how faithfully model outputs are grounded in the provided evidence. We present a controlled study of hallucination behavior under context compression and introduce Attention-Based Semantic Preservation (ABSP), a lightweight query-agnostic method that retains tokens according to structural centrality in a proxy model’s self-attention patterns. Across 2,400 inference runs comparing no compression, random pruning, LLMLingua-2, and ABSP on TriviaQA and HotpotQA, structural compression consistently improves grounding faithfulness, measured by K-Precision. ABSP achieves the highest K-Precision among compressed methods at every compression ratio, outperforming LLMLingua-2 by + 0.111 on average and + 0.131 under aggressive compression, while matching its answer accuracy on single-hop retrieval. Cross-sample consistency is stable across methods, suggesting this hallucination signal is driven more by the generation model than by the compression strategy.