Stepwise-Querying: Distilling Reasoning without Access to Teacher Chain-of-Thought
This work proposes Stepwise Querying, which elicits step-specific explanations without requiring access to the teacher's internal chain-of-thought (CoT) traces, and introduces Stepwise Explanation Information Gain (SEIG), a diagnostic based on the log-ratio of a frozen student's probabilities of correct step completion...