Structured Hallucinations in Small Language Models: Repeatability, Attention Signatures, and Context Contagion
Abstract
In this paper, we present an empirical analysis of hallucination behavior in small language models (1.5B–7B parameters), addressing the question of whether such failures are stochastic or structured. Our analysis of 4,500 responses from three instruction-tuned models, evaluated on TruthfulQA and HaluEval, reveals that hallucinations exhibit highly consistent, non-random patterns. First, 92.85% of question–model pairs that hallucinate do so consistently across repeated runs (χ2 =4316.4, p < 0.001), indicating highly repeatable failure modes tied to specific queries. Second, layer-wise attention entropy reveals a consistent mechanistic signature: hallucinated responses exhibit significantly higher early-layer entropy in two distinct architectures (LLaMA: t = 13.305; Mistral: t = 8.976; p < 0.001). Third, the introduction of false contextual information increases the hallucination rate by 1.69× (p = 0.0003), with modelspecific amplification effects reaching up to 3.06×. Furthermore, hallucinated responses exhibit model-specific lexical signatures. A TF-IDF classifier achieves > 86.75% accuracy in identifying the source model, substantially outperforming semantic embedding-based methods (63.11%). Together, these findings demonstrate that hallucinations in small language models are structured and reproducible, underscoring the need for targeted mitigation strategies rather than reliance on scaling alone.