A D-LLM hallucination detection framework that formulates the Denoising trajectories as Multivariate Time Series over learnable latent variables (DeMTS for short) that outperforms existing hallucination detection methods while maintaining strong robustness, efficiency, and cross-task transferability.
Abstract
Diffusion large language models (D-LLMs) have emerged as a promising paradigm for text generation. However, similar to autoregressive LLMs, D-LLMs remain vulnerable to hallucinations, where fluent outputs may contain factually incorrect or unsupported content. Although existing hallucination detection methods for D-LLMs attempt to leverage uncertainty trajectories of the denoising process to better identify hallucination signals, they typically compress the trajectories along either the temporal or token dimension, overlooking the useful information encoded in the complete two-dimensional token-step structure. Consequently, they may fail to capture hallucination-relevant patterns, such as inconsistent convergence and cross-token fault propagation, leading to suboptimal detection performance. To bridge this gap, we propose a D-LLM hallucination detection framework that formulates the Denoising trajectories as Multivariate Time Series over learnable latent variables (DeMTS for short). DeMTS employs a trajectory-preserving token-to-variable assignment module to convert token signals into stable latent variables. Based on these variables, we propose dynamic multivariate temporal modeling to progressively integrate inter-variable dependency modeling with temporal encoding for hallucination prediction. Extensive experiments on two D-LLMs backbones and three benchmarks demonstrate that DeMTS outperforms existing hallucination detection methods while maintaining strong robustness, efficiency, and cross-task transferability.
Large Language Models (LLMs) are increasingly being applied to time-series forecasting, giving rise to a class of models referred to as time-series LLMs. While these models achieve competitive predictive accuracy, their reliability and structural consistency remain insufficiently understood. These models may produce forecasts that are numerically accurate on average yet statistically or temporally implausible, a phenomenon referred to as hallucination. Unlike conventional forecasting errors, hallucinations represent deviations from underlying temporal dynamics that exceed expected volatility patterns. This paper presents a systematic investigation of hallucination in time-series LLM forecasting. We introduce a quantitative evaluation framework that complements traditional regression metrics with two reliability-oriented measures: perplexity (PP), which reflects predictive uncertainty, and hallucination rate (HR), which measures statistically significant deviations from ground truth. Experiments on widely used benchmark datasets (Electricity and ETT variants) reveal a critical trade-off between forecasting accuracy and reliability. In several settings, improvements in average error metrics do not correspond to improved reliability; models can maintain low mean absolute error (MAE) while exhibiting high HR. To mitigate this issue, we evaluate two strategies: data-centric preprocessing, whose effectiveness depends on dataset characteristics, and structured tokenization, which consistently reduces hallucination across the evaluated datasets. Sensitivity analysis over quantile thresholds confirms the robustness of hallucination trends and model rankings. These results demonstrate that conventional regression metrics alone are insufficient for evaluating time-series LLMs and highlight the need for reliability-focused diagnostics when deploying LLM-based forecasting systems in high-stakes domains.
Shamsu Abdullahi, K. U. Danyaro, H. Chiroma et al.· Scientific Reports· 0 citations
The D-Score is introduced, a simple spectral statistic computed from a single forward pass that is used as a hallucination score, classifying an input text as hallucinated when its D-Score is larger than a pre-defined quantity.
Bianca Raimondi, Davide Evangelista, Maurizio Gabbrielli et al.· 0 citations
InnerExpert is introduced, the first method to leverage MoE-specific signals for per-token hallucination detection, and shows that InnerExpert outperforms existing methods across five datasets and two MoE architectures.
João Fonseca, Rodrigo Rodrigues, Paolo Romano· 0 citations
Large Language Models (LLMs) tend to hallucinate when processing symbolically complex linguistic structures. Existing literature evaluates hallucination either through their mechanistic interpretability or at the behavioral output level, but hardly links the symbolic triggers to their layer-wise representational causes. This paper introduces a unified symbolic, behavioral, and mechanistic framework that connects symbolic triggers with internal failure dynamics in transformer architectures. The study evaluates five open-weight LLMs across QA, MCQ, and Odd-One-Out formats on the HaluEval and TruthfulQA datasets, focusing on negation, exceptions, modifiers, numbers, and named entity cues. The results show that hallucination rates remain high across all model scales, with all symbolic categories exhibiting high hallucination rates, and exceptions and numbers often showing comparable or higher values across models. Constrained task formats reduce surface errors but preserve failure patterns, indicating representational instability rather than purely decoding artifacts. Layer-wise analysis shows peak symbolic attention variance in early transformer layers (2–4), after which these patterns persist across deep layers. The consistency of this behavior across architectures suggests that hallucination is strongly associated with weakness in symbolic encoding. The framework provides an interpretable basis for diagnosing and stabilizing symbolic reasoning in LLMs.
Latent Attention Denoising (LAD) is introduced, a principled and training-free framework that recasts attention calibration as a one-step score-based denoising process that achieves superior performance on both generative and discriminative tasks.
Zhiwen Luo, Siyu Jiang, Weilong Jiang et al.· Annual Meeting of the Associ...· 0 citations
GHOST is proposed, an efficient white-box framework for hallucination detection in LLMs that primarily target confused hallucinations marked by internal reasoning instability, while also capturing stub-born hallucinations characterized by premature layer-wise convergence as a complementary signal.
Yueheng Mao, Min Yu, Gengwang Li et al.· Annual Meeting of the Associ...· 0 citations