Jun 2026· 2026 IEEE 2nd International Conference on Electronics, Energy Systems and Power Engineering (EESPE)· pp. 2176-2181· 0 citations· 13 references
Abstract
Addressing the bottlenecks of existing Chain-of-Thought (CoT) methods in complex reasoning of Large Language Models (LLMs), such as reasoning skips, frequent logical illusions, and unbalanced computational costs, this paper proposes an original Adaptive Bidirectional Verification Chain-of-Thought (ABV-CoT) algorithm. This algorithm comprises three core modules: a task complexity-aware forward adaptive reasoning step size generation module, which dynamically matches reasoning granularity with task difficulty; a reverse logical closed-loop verification and error backtracking correction module, which can locate and correct intermediate reasoning errors, reducing logical illusions and error accumulation; and a lightweight gated attention optimization unit, which can improve reasoning performance while controlling computational costs. Comparative and ablation experiments are conducted on four standard complex reasoning datasets, including GSM8K and MATH, using Llama-2-7B, Llama-2-13B, and Qwen-14B as base models. The results show that the ABV-CoT algorithm improves the average inference accuracy by 5.12-19.78 percentage points compared to the mainstream CoT method, reduces inference latency by 36.84% compared to the Tree-of-Thoughts (ToT) method, and reduces memory usage by 26.09%. It exhibits excellent generalization ability on small-to-medium-scale models, providing a new technical path and theoretical support for enhancing the complex reasoning capabilities of LLMs, and has significant academic and engineering value.
Chain-of-Thought (CoT) reasoning has significantly enhanced the multi-step problem-solving capabilities of large language models (LLMs) by introducing explicit intermediate reasoning. However, advanced Large Reasoning Models (LRMs) often exhibit overthinking behaviors, including excessively long reasoning steps, redundant steps, and high computational overhead. Existing token-length reward strategies aim to promote concise outputs, but often result in pseudo-conciseness, where token count is reduced, yet redundant reasoning persists, leading to longer and less structurally efficient chains. To address these limitations, we propose ChainPrune, a novel reasoning path semantic structural optimization method to efficiently and controllably synthesize self-generated high-quality training data. We initially consolidate self-generated reasoning paths into a tree-based structure, followed by a multi-criteria dominant path selection process for preference data construction that formulates shallow reasoning trajectories while preserving essential reasoning steps. To further enhance the quality of reasoning, we incorporate a DPO-based preference learning method combined with supervised loss, effectively mitigating false reward suppression. This innovative integration significantly enhances both the efficiency and effectiveness of our reasoning framework. Comprehensive experimental results demonstrate significant reductions in step length and computational overhead, while maintaining or even enhancing accuracy.
Weihang Pan, Zhengxu Yu, Yuxiang Zhang et al.· 1 citation
Reason Popper-ly, a neurosymbolic framework that uses inductive logic programming (ILP) to learn relation composition rules from reasoning traces and deploys them as an online verifier for step-level correction, consistently improves terminal accuracy over standard CoT.
This work presents a theoretical framework that reveals how reasoning steps can amplify error through three failure modes: incorrect sub-task decomposition, incorrect sub-task solving, and incorrect final answer summarization, and introduces structured interventions that adapt CoT generation according to the identified failure types.
Haibo Jin, Peiyan Zhang, Man Luo et al.· Neural Information Processin...· 1 citation
Large language models often rely on Chain-of-Thought (CoT) reasoning to solve complex tasks, but verbose reasoning traces introduce substantial inference overhead. CoT compression shortens generation, yet aggressive compression may disrupt logical coherence and degrade performance. We formalize this trade-off as the Context-Generation Substitution Law, where explicit reasoning context substitutes for part of decode-time generation. Based on this principle, we propose Memory-Augmented Compression, a training-free framework that constructs reusable reasoning memories from historical traces and retrieves them as prefill-side scaffolds. Rather than using raw demonstrations, these memories summarize reusable reasoning patterns, key constraints, and critical operations to compensate for information lost during compression. Experiments show that Memory consistently improves prompt-based Chain-of-Draft (CoD) compression across mathematical reasoning, complex reasoning, and science question answering tasks, yielding accuracy gains of 21.4, 28.0, 29.5, and 6.61 points over CoD on GSM8K, MATH, BBH, and MMLU-Sci, while achieving a 1.14-1.49x latency speedup latency speedup over standard CoT. Memory is also compatible with token-level, reasoning-trace-level, and inference-state compression mechanisms.
Simeng Zhang, Yilong Chen, Wenyuan Zhang et al.· 0 citations
Experimental results show that PD-CoTD significantly enhances the mathematical reasoning abilities of SLMs and provides an effective solution for deploying powerful reasoning in resource-constrained environments.
Zhe Wang, Long Zhang, Fengbo Zheng et al.· International journal of sof...· 0 citations
OS-Pruner is a lightweight plug-in framework that formulates chain-of-thought pruning as an optimal stopping problem that achieves 20-60\% reduction in generation length with minimal accuracy sacrifice on diverse reasoning benchmarks and base models.
Mohammed Ehab, Aymane El Gadarri, Vivek F. Farias et al.· 0 citations