Structured Hallucinations in Small Language Models: Repeatability, Attention Signatures, and Context Contagion
In this paper, we present an empirical analysis of hallucination behavior in small language models (1.5B–7B parameters), addressing the question of whether such failures are stochastic or structured. Our analysis of 4,500 responses from three instruction-tuned models, evaluated on TruthfulQA and HaluEval, reveals that...