2026· IEEE Transactions on Audio, Speech, and Language Processing· Vol 34, pp. 4510-4525· 0 citations· 50 references
TL;DR
This paper proposes LLM Introspection, a practical inference-time reasoning control method that leverages keyword-token logits to enable early stopping during the reasoning process, and integrates the method into a vLLM-based proxy, providing a practical, plug-and-play deployment path.
Abstract
Recent Reasoning Large Language Models have demonstrated impressive long-form reasoning capabilities, achieving expert-level performance on complex tasks such as competition-level mathematics and scientific question answering. However, they often suffer from the overthinking problem, leading to excessive token usage, higher latency, and an increased risk of hallucinations over long contexts. Existing solutions often require costly post-training of the underlying LLM or incur extra decoding overhead from probing intermediate answers. In this paper, we propose LLM Introspection, a practical inference-time reasoning control method that leverages keyword-token logits to enable early stopping during the reasoning process. By introspecting the model’s internal confidence state through a lightweight GRU-based module operating on the token-logit stream in the reasoning phase, our method dynamically terminates reasoning once sufficient confidence is reached, effectively mitigating overthinking. Experiments on Qwen3 and DeepSeek-R1-Distill across MATH-500, GSM8K, AIME, and GPQA Diamond show consistent token savings with comparable accuracy in most cases. On Qwen3-4B, token reductions range from 4.3% to 41%, with generally improved Accuracy–Efficiency Scores. We further integrate our method into a vLLM-based proxy, providing a practical, plug-and-play deployment path.
Experiments show that CALM reduces token consumption by 18.6%-25.0% compared with full CoT reasoning while maintaining performance, and can even improve accuracy by 0.07%-1.48% by reducing redundant reasoning.
Sheng-Jing Liu, Fei-Yang Li, Yuxuan Yang et al.· 0 citations
This paper proposes a novel training-free framework to achieve efficient reasoning that reduces token generation costs without sacrificing performance, and investigates the latent representations and proposes a novel training-free framework.
Large language models (LLMs) are increasingly described as acquiring"reasoning"skills after reinforcement learning with verifiable rewards. However, whether these gains reflect genuine reasoning or more superficial adaptations remains underexplored. In this work, we probe LLMs trained in a finite and exhaustively analy...
Comparing three model families and multiple math benchmarks, COMPASS outperforms the activation-steering baselines the authors compare against, improves GSM8K accuracy by 16 percentage points on average, and approaches CoT accuracy with 20-70\% fewer generated tokens.
Pratyay Dutta, Kowshik Thopalli, V. Narayanaswamy· 0 citations
Decision-Flow Sampling (DF-Sample), a training-free, data-free inference-time framework that constructs a hierarchical reasoning tree, scores terminal nodes for quality, and back-propagates utilities to inform each intermediate branching decision, indicating substantial latent reasoning potential in pretrained base mod...
Despite their extensive pretraining knowledge and strong reasoning capabilities, large language models (LLMs) struggle to adaptively gather information in experimental design tasks. We study how to amortize sequential Bayesian experimental design (BED) into an LLM by explicitly training the reasoning that guides action...
Anita Kriz, Antonin Berthon, Nicolás Astorga et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.