Jul 2026· International journal of software engineering and knowledge engineering· 0 citations
TL;DR
Experimental results show that PD-CoTD significantly enhances the mathematical reasoning abilities of SLMs and provides an effective solution for deploying powerful reasoning in resource-constrained environments.
Abstract
Large Language Models (LLMs) have strong capabilities in mathematical and logical reasoning. However, deploying LLMs is difficult for users with limited hardware. One solution is distilling reasoning skills from LLMs to Small Language Models (SLMs, ≤1B parameters). Still, student models often make five common errors: calculation errors, missing reasoning steps, symbol mistakes, semantic misunderstandings, and coherence errors.This paper proposes Prefix-Description-CoTD (PD-CoTD), an improved distillation method to address these issues. First, structured prompt templates guide LLMs to generate complete reasoning steps. This creates high-quality datasets to help SLMs learn effective reasoning patterns. Second, we introduce a step-by-step optimization loss function. This loss function separates reasoning path modeling from final answer prediction. It helps the model focus on reasoning clearly while optimizing results. Experimental results show that PD-CoTD significantly enhances the mathematical reasoning abilities of SLMs. It achieves state-of-the-art performance and greatly reduces the five common errors. PD-CoTD provides an effective solution for deploying powerful reasoning in resource-constrained environments. It also supports the broader application of lightweight models in academic research and education. Our code implementation and data are available at https://github.com/gameking2159/PD-CoTD .
It is suggested that using standard CoT prompting increasingly acts as a source of distraction as models grow stronger, because standard CoT prompting also demands style adaptation, formatting compliance, and potentially undesired contextualization, which can distract models from the core reasoning task.
Denys Pushkin, Albert Q. Jiang, Aryo Lotfi et al.· 0 citations
Reason Popper-ly, a neurosymbolic framework that uses inductive logic programming (ILP) to learn relation composition rules from reasoning traces and deploys them as an online verifier for step-level correction, consistently improves terminal accuracy over standard CoT.
This work proposes Step-wise Training for In-context Reasoning (STIR), a model to dynamically decide when to retrieve a single logically consistent next step, just using the current problem and its intermediate state as the query.
Cheng Yang, Zhenya Huang, Liyang He et al.· Proceedings of the 32nd ACM...· 0 citations
Large Language Models (LLMs) are predominantly assessed based on their common sense reasoning, language comprehension, and logical reasoning abilities. While models trained in specialized domains like mathematics or coding have demonstrated remarkable advancements in logical reasoning, there remains a significant gap in evaluating their code generation capabilities. Existing benchmark datasets fall short in pinpointing specific strengths and weaknesses, impeding targeted enhancements in models’ reasoning abilities to synthesize code.
To bridge this gap, this thesis introduces two novel contributions: CodeEval and CodeQual. CodeEval is an innovative, pedagogical benchmarking method that mirrors the evaluation processes encountered in academic programming courses. It comprises a multi-dimensional benchmark dataset of 602 hand-crafted problems designed to rigorously evaluate LLMs across 24 distinct aspects of Python programming, covering three proficiency levels—beginner, intermediate, and advanced—and includes both class-based and function-based problem types with detailed problem specifications and comprehensive test suites achieving 99.1% coverage. To facilitate widespread adoption, we developed RunCodeEval, an open-source execution framework that provides researchers with a ready-to-use evaluation pipeline. Our evaluation of 15 state-of-the-art LLMs revealed consistent performance degradation with increasing complexity (validated statistically, Cohen’s d = 0.790) and universal struggles with advanced concepts like concurrency.
Code quality is inherently subjective, encompassing dimensions like readability, efficiency, and adherence to language idioms that traditional static metrics fail to capture adequately. While large language models can assess these subjective qualities, lightweight models offer practical advantages: seamless CI/CD pipeline integration, lower operational costs, and full control over model behavior. We investigate whether such models can learn to assess code quality by training on synthetic LLM annotations. We introduce CodeQual, a dataset of 5,819 code samples derived from five established sources spanning diverse domains—competitive programming, pedagogical problems, software engineering, and general benchmarks—scored by LLMs across five quality dimensions, with 655 human-annotated samples for evaluation. Our fine-tuned model, CodeQualBERT, not only matches LLM performance but exceeds inter-human agreement on all five dimensions, achieving 16–100% improvement over the inter-human agreement baseline.
Together, these contributions provide a comprehensive framework for evaluating and improving LLMs in software engineering contexts, encompassing both functional correctness assessment and subjective code quality evaluation.
This work conducts a fine-grained analysis of hallucinations arising in LLM reasoning and finds that the reasoning traces are particularly prone to Context-Sensitive Factual Hallucinations: cases where the model actually has the relevant knowledge, yet makes factual errors due to contextual interference during reasoning.
Xiaomeng Hu, Jiaqi Hu, Hao Chen et al.· 0 citations
Large Language Models (LLMs) demonstrate significant potential in sequential recommendation, and leveraging their Chain-of-Thought (CoT) reasoning capabilities can further unlock profound user preference understanding. However, deploying explicit CoT reasoning in real-world systems faces prohibitive challenges: (i) the conflict between the large model scale required for high-fidelity reasoning and the resource constraints of online services, and (ii) the excessive latency introduced by auto-regressive rationale generation. To address these issues, we propose I Reasoning via Multi-Teacher Distillation (IRMD), a novel framework that 'compiles' the reasoning abilities of large teacher LLMs into a lightweight student Small Language Model (SLM). IRMD first employs a Multi-Teacher CoT Synthesis with Dual-Constraint Rejection Sampling module to generate a high-quality, diverse set of reasoning paths. Subsequently, our Annealing-Scheduled Reasoning Distillation strategy progressively trains the student to internalize this logic, transitioning from mimicking explicit CoT to performing purely implicit reasoning. Extensive experiments on multiple benchmark datasets demonstrate that IRMD significantly outperforms state-of-the-art baselines in both recommendation accuracy and inference efficiency. Our code is accessible at https://github.com/Cxx-0/IRMD.