Skip to content

From Tokens to Semantics: Leveraging Complementary Signals for Hallucination Detection in Black-Box LLMs

Sep 2026 · arXiv.org · Vol abs/2609.02679 · 0 citations · 32 references
Computer Science

TL;DR

In the evaluation across models and datasets, Stacked gave the best performance in nearly half of the cases, while TopK and CoCoA remain competitive without supervised training labels, although their thresholds require careful calibration.

Abstract

When LLMs support public-facing or high-stakes workflows, missed fabrications can harm users and institutions, while false alarms consume limited human-review capacity. When no trusted context or reference document is available, we study two signals accessible through black-box model APIs: semantic entropy, which measures disagreement among sampled response meanings, and uncertainty derived from token log-probabilities. Their failure modes can be complementary: semantic entropy becomes uninformative when responses form one semantic cluster, while token uncertainty can miss consistently confident errors. We extend token-based uncertainty detection by aggregating token-level signals across sampled responses through our TopK method, evaluate the hybrid CoCoA method, which combines target-response uncertainty with semantic dissimilarity, and propose and study two supervised methods: Gated, which routes single-cluster cases to an aggregated-token-feature classifier, and Stacked, which learns jointly from semantic uncertainty and broader token features. We evaluate seven benchmarks, including five public benchmarks (four text datasets and multimodal handwritten-cheque extraction) and two constructed benchmarks (Financial Summaries and Long-Text QA), using four language models. In our evaluation across models and datasets, Stacked gave the best performance in nearly half of the cases, while TopK and CoCoA remain competitive without supervised training labels, although their thresholds require careful calibration. No method is universally strongest. We therefore evaluate performance at false-positive-rate budgets from 1% to 15%, assess their sensitivity to generation and calibration choices, and examine variation across dataset characteristics.

View source

Similar papers

#natural language process... Preprint Oct 2026

Gray-Box Hallucination Detection in LLMs with Token Identities

GURI (Gray-box Uncertainty Representations with Identity), a lightweight gray-box hallucination detector that combines candidate-token probabilities with per-token vector identifiers that exposes which alternatives the model considered while preserving the accessibility and transferability of the gray-box setting.

S. Goldshtein, Yitzchak Grunbaum, Fabrizio Frasca et al. · 0 citations
2026

Semantic Entropy as a Scalable Hallucination Detection Signal in Large Language Models: Theoretical Grounding, Calibration, and Downstream Task Benchmark

These findings contribute empirically to machine learning and natural language processing and carry actionable implications for the design of programs and policies targeting LLM-powered QA system reliability gating, medical/legal hallucination risk reduction, and uncertainty-aware RAG system design.

James T. Wu, Anika M. Johansson, Dele K. Ogunyemi · 0 citations
Preprint Nov 2026

From Twisted Facts to Rogue Agents: Can LLMs Reliably Detect Multi-Dimensional Hallucinations?

HaluCheck, a large-scale, bilingual benchmark comprising over 42k instances spanning both factual hallucinations in text and faithfulness hallucinations in agent trajectories, provides a solid basis for benchmarking and diagnosing LLM reliability across text and agent scenarios.

Han-Yu Qiu, Ze-Li Liu, Zhe Zhao et al. · 0 citations
Preprint Oct 2026

LLM Hallucination Detection via Low-rank Multilinear Decomposition

Large language models (LLMs) frequently generate fluent yet factually incorrect responses, raising concerns about their reliability. Recent hallucination detectors increasingly exploit internal representations, but extracting informative signals from high-dimensional hidden states remains challenging. Existing approach...

Devansh Sharma, Faizul Rakib Sayem, Shahana Ibrahim · 0 citations
#artificial intelligence Preprint Oct 2026

CARLAR Influence: A Structured Hallucination Risk in Multimodal Cheque Extraction and an Exploratory Compressed-Image Countermeasure

Multimodal large language models (LLMs) are increasingly used for structured extraction from financial documents, but their behavior under visual ambiguity is poorly understood. We study CARLAR (Courtesy Amount Recognition and Legal Amount Recognition) influence: a model returns the same amount for both cheque fields e...

J. Choi, Aditya Paul, Nidhi Singh · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.