Skip to content
Open access

Efficient LLM Reasoning via Logit-Based Introspection

2026 · IEEE Transactions on Audio, Speech, and Language Processing · Vol 34, pp. 4510-4525 · 0 citations · 50 references

TL;DR

This paper proposes LLM Introspection, a practical inference-time reasoning control method that leverages keyword-token logits to enable early stopping during the reasoning process, and integrates the method into a vLLM-based proxy, providing a practical, plug-and-play deployment path.

Abstract

Recent Reasoning Large Language Models have demonstrated impressive long-form reasoning capabilities, achieving expert-level performance on complex tasks such as competition-level mathematics and scientific question answering. However, they often suffer from the overthinking problem, leading to excessive token usage, higher latency, and an increased risk of hallucinations over long contexts. Existing solutions often require costly post-training of the underlying LLM or incur extra decoding overhead from probing intermediate answers. In this paper, we propose LLM Introspection, a practical inference-time reasoning control method that leverages keyword-token logits to enable early stopping during the reasoning process. By introspecting the model’s internal confidence state through a lightweight GRU-based module operating on the token-logit stream in the reasoning phase, our method dynamically terminates reasoning once sufficient confidence is reached, effectively mitigating overthinking. Experiments on Qwen3 and DeepSeek-R1-Distill across MATH-500, GSM8K, AIME, and GPQA Diamond show consistent token savings with comparable accuracy in most cases. On Qwen3-4B, token reductions range from 4.3% to 41%, with generally improved Accuracy–Efficiency Scores. We further integrate our method into a vLLM-based proxy, providing a practical, plug-and-play deployment path.

Read PDF

Similar papers

Efficient Reasoning via Constrained Optimization in Latent Space

This paper proposes a novel training-free framework to achieve efficient reasoning that reduces token generation costs without sacrificing performance, and investigates the latent representations and proposes a novel training-free framework.

Zhi-Nan Hou, Xing-Chen Li, Ke-You You · 2 citations
#natural language process... Preprint Oct 2026

Reasoning or Pattern Exploitation? Mechanistic Insights from RL-Trained Language Models

Large language models (LLMs) are increasingly described as acquiring"reasoning"skills after reinforcement learning with verifiable rewards. However, whether these gains reflect genuine reasoning or more superficial adaptations remains underexplored. In this work, we probe LLMs trained in a finite and exhaustively analy...

Aishik Nagar, Cheston Tan · 0 citations
#small language model Preprint Oct 2026

COMPASS: Finding Where Reasoning Lives in Language Models

Comparing three model families and multiple math benchmarks, COMPASS outperforms the activation-steering baselines the authors compare against, improves GSM8K accuracy by 16 percentage points on average, and approaches CoT accuracy with 20-70\% fewer generated tokens.

Pratyay Dutta, Kowshik Thopalli, V. Narayanaswamy · 0 citations
#machine learning Preprint Sep 2026

Sampling via Decision-Flow: Training-Free Extraction of Improved Latent Reasoning Paths in Large Language Models

Decision-Flow Sampling (DF-Sample), a training-free, data-free inference-time framework that constructs a hierarchical reasoning tree, scores terminal nodes for quality, and back-propagates utilities to inform each intermediate branching decision, indicating substantial latent reasoning potential in pretrained base mod...

Zhen-Dong Mi, Shao-Yi Huang · 0 citations
#artificial intelligence Preprint Oct 2026

Amortizing Sequential Experimental Design in Reasoning LLMs

Despite their extensive pretraining knowledge and strong reasoning capabilities, large language models (LLMs) struggle to adaptively gather information in experimental design tasks. We study how to amortize sequential Bayesian experimental design (BED) into an LLM by explicitly training the reasoning that guides action...

Anita Kriz, Antonin Berthon, Nicolás Astorga et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.