Demystifying Entropy-based Selection for Chain-of-Thought Compression in Large Reasoning Models
It is demonstrated that patching a subset of a few CoT tokens with their original activations recovers near-perfect full-trace performance, providing causal evidence that task information is not concentrated in a small set of CoT tokens identifiable by heuristics, but rather distributed across the full reasoning chain.