Skip to content
Preprint

From Twisted Facts to Rogue Agents: Can LLMs Reliably Detect Multi-Dimensional Hallucinations?

Nov 2026 · 0 citations · 39 references
Computer Science

TL;DR

HaluCheck, a large-scale, bilingual benchmark comprising over 42k instances spanning both factual hallucinations in text and faithfulness hallucinations in agent trajectories, provides a solid basis for benchmarking and diagnosing LLM reliability across text and agent scenarios.

Abstract

Large language models (LLMs) are widely used for information access, content production, and agent-based task execution, but their outputs remain vulnerable to subtle, high-risk hallucinations. To systematically evaluate and advance post-generation hallucination detection, we introduce HaluCheck, a large-scale, bilingual benchmark comprising over 42k instances spanning both factual hallucinations in text and faithfulness hallucinations in agent trajectories. HaluCheck features fine-grained, multi-level annotations for detection, localization, and correction, covering different tasks and perturbation degrees. Experiments on fourteen LLMs reveal that, while models show basic sample-level detection ability, they struggle with fine-grained localization, correction, and agent hallucination detection. Further analysis demonstrates that thinking, web search, and supervised fine-tuning each offer partial improvements, but none fully solve the problem. HaluCheck thus provides a solid basis for benchmarking and diagnosing LLM reliability across text and agent scenarios.

View source

Similar papers

Conference Oct 2026

HallGraph: A Zero-Resource Approach for Hallucination Detection in LLMs

Hallucination in large language models, the generation of fluent but factually unsupported text, remains a barrier to their safe deployment. Existing zero-resource detectors such as SelfCheckGPT measure consistency across stochastic samples but treat all claims uniformly and draw a fixed, often excessive, number of gen...

Tetala Rukmender Reddy, Vedant Mathur, Tanay Aggarwal et al. · 0 citations

Hallucination Neurons and Where to Find Them: An Investigation into the existence of Hallucination Neurons

This work proposes a five-step diagnostic protocol covering feature correlation, bootstrap stability, sparse versus dense ranking disagreement, intervention baselines, and cross-dataset evaluation as a minimum standard for sparse-neuron localization claims, and investigates prior work using the proposed approach.

Huseyin Cavus, Sebin Sabu, J. Spear et al. · 0 citations
Open access Sep 2026

Mitigating Hallucinations in Finance-Based Multi-Agent Large Language Model Systems

This research proposes a multi-signal framework for hallucination detection and mitigation that corrects up to 78.4% of the detected hallucinations and highlights that targeted, type-aware hallucination mitigation can significantly improve answer reliability while remaining computationally effective.

Rashmi Nagpal, Unyimeabasi Usua, Kailey Simons et al. · 0 citations
Conference Oct 2026

Structured Hallucinations in Small Language Models: Repeatability, Attention Signatures, and Context Contagion

In this paper, we present an empirical analysis of hallucination behavior in small language models (1.5B–7B parameters), addressing the question of whether such failures are stochastic or structured. Our analysis of 4,500 responses from three instruction-tuned models, evaluated on TruthfulQA and HaluEval, reveals that...

Kushagra Agrawal, Yu-Ming Feng, Hang-Jun Che et al. · 0 citations
#artificial intelligence Preprint Oct 2026

SOPO: Socratic Guided Policy Optimization for Span-Level Hallucination Detection

The key innovation of SOPO is a Socratic feedback mechanism, where a teacher model guides the reasoning trajectories during rollout by providing indirect hints inferred from the ground truth, enabling the model to generate more effective reasoning trajectories for hard examples, thus alleviating reward sparsity.

Jia-Wei Dong, Zi-Long Bai, Peng-Tian Zhu et al. · 0 citations
Conference Sep 2026

Hallucination Detection and Reduction in Genai (LLMs)

Hallucinations are false or unverifiable outputs generated by Large Language Models (LLMs) that reduce their trustworthiness in critical applications such as healthcare, education, and research. This paper proposes a hybrid framework that unifies three complementary verification signals for the detection and reduction...

Radhika K. R., Sumit Gupta, Nagesh H. Shenoy et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.