Negative Contrastive Chain-of-Thought Distillation for Transfer Reasoning Capability to Small Language Models

Abstract

Large language models (LLMs) demonstrate strong reasoning capabilities through chain-of-thought prompting. Recently, numerous studies have focused on transferring reasoning capability from LLMs to small language models (SLMs) through chain-of-thought (CoT) distillation. However, these approaches face two limitations. First, simplistic multiple iterations to augment correct predictions are ineffective, due to the LLM tends to reproduce the same errors. Second, directly utilizing incorrect predictions during the training process exposes the model to learning incorrect predictions of the LLM during distillation. To address these problems, we propose a novel framework named negative contrastive CoT distillation (NCD), which efficiently augments correct predictions while training an SLM to exclude incorrect predictions. NCD comprises of two stages: corrective reviewer augmentation (CRA) and negative exclusive learning (NEL). CRA utilizes the self-correction capability of LLM to augment the correct predictions within a single iteration. NEL is a distillation strategy that strengthens the ability of SLM to mimic the correct predictions of LLM by leveraging the incorrect predictions of LLM. The experimental results show that NCD improves the reasoning capability of the SLM by a sizable gap in diverse reasoning tasks, including mathematical and commonsense reasoning tasks. We demonstrate both the efficiency of CRA and the effectiveness of NEL.