Training a Generalist Hallucination Detector across Multiple Domains via Adaptive Layer Aggregation

Abstract

Hallucination detection is essential for the reliable deployment of large language models (LLMs) in safety-critical applications, yet existing detectors often degrade under real-world domain shifts. To formalize this challenge, we introduce Multi-Domain Generalizable Hallucination Detection (MGHD), where a detector is trained on multiple labeled domains and evaluated on unseen domains without any test-domain supervision. The central challenge of MGHD stems from a previously underexplored phenomenon, termed layer-index shift, in which the most informative internal layer varies substantially across domains, causing detectors that rely on a fixed layer to generalize poorly. To mitigate this phenomenon, we propose a novel method Stability-Aware Adaptive Layer Aggregation (SALA). SALA defines a layer-wise instability index computed from the training data, which measures how separability varies across domains. For an unseen test domain, it provides an upper bound on the possible separability drop from training to test. Guided by this bound, SALA designs a per-layer projector that balances discriminability and stability. Then using these projectors, SALA learns invariant subspaces across layers and aggregates them into a robust detector. Experiments across four widely used QA benchmarks and four LLM backbones from the LLaMA and Qwen families (7B-14B) show that SALA achieves the best average performance compared with existing hallucination detection baselines.