KGA-LM: Representation-Level Grounding for Conversational Search over Knowledge Graphs

Abstract

Integrating structured knowledge graphs (KGs) with Large Language Models (LLMs) is essential for trustworthy, knowledge intensive conversational systems. However, existing Retrieval Augmented Generation (RAG) methods typically rely on a retrieval-as-context paradigm that linearizes structured subgraphs into unstructured prompt tokens. This approach not only flattens rich structural dependencies but also leads to context inflation and evidence attenuation in multi-turn dialogues. To address these limitations, we propose KGA-LM, a framework that integrates external knowledge via representation-level grounding. Rather than treating retrieved evidence as transient input artifacts, KGA-LM encodes compact multi-hop subgraphs using a Graph Transformer and fuses them into the LLM decoder through a compatibility-aware latent interface. This design aligns the heterogeneous latent spaces of the graph encoder and the LLM, while a dual-gated fusion mechanism dynamically regulates the influence of non-parametric graph evidence across turns. Experiments on multiple conversational benchmarks demonstrate that KGA-LM significantly improves factual accuracy and reduces hallucination compared to prompt-linearized baselines. Crucially, by decoupling knowledge injection from prompt length, our approach mitigates retrieval signal decay under long contexts, offering a superior trade-off between grounding quality and inference efficiency.