These findings establish correct-solution retention as a complementary objective of recursive self-improvement (RSI) and show that generation-time intervention can improve the solution repertoire retained by subsequent students.
Abstract
A model that learns from its own outputs inherits more than their correctness: it inherits which solutions it produces. We formulate Looped Self-Distillation, a self-evolution framework for code generation in which a model repeatedly generates and learns from its own raw outputs, under a fixed information budget, without ongoing external assessment or test-based selection of the generated samples. We identify a consequential separation: correctness can improve while the breadth of correct implementations contracts. We introduce SPECTRUM, which re-estimates loss-sensitive key/value geometry from a fixed reference anchor at each round and converts it into full-rank proximal spectral modulation. All generated completions train a single student, whose subsequent inference requires no intervention. After five rounds of experiments on MBPP, SPECTRUM retains 89.9% of the initial model's 64-sample correct AST richness, compared with 66.4% for Vanilla self-distillation and 65.5% for a subspace-projection control. The advantage persists at matched correct-sample counts. Without further training or recalibration, the resulting student also achieves higher matched-correct richness than Vanilla SD on HumanEval+ and APPS Intro, demonstrating transfer of the diversity benefit. These findings establish correct-solution retention as a complementary objective of recursive self-improvement (RSI) and show that generation-time intervention can improve the solution repertoire retained by subsequent students.
Activation-Conditioned Self-Distillation (ACSD), which extracts a steering vector by contrasting activations of self-generated trajectories that reach verified correct answers within a generation budget with those of all remaining trajectories, achieves the highest mean accuracy over four mathematical benchmarks among...
This paper observes that when a safe model is recursively self-distilled on its own (safe) generations while receiving an additional non-harmful cue, it becomes increasingly misaligned across a diverse set of harmful queries; it is called Progressive Misalignment.
Karim Abdel Sadek, Federico Di Gennaro, Fanny Yang· 0 citations
Controling how much and when teacher information is introduced improves specialisation while preserving broader capability is shown, and results show that controlling how much and when teacher information is introduced improves specialisation while preserving broader capability.
Ahmed Khaled Khamis, Xiao-Tong Ji, Hassan Jaber et al.· arXiv.org· 1 citation
On-policy self-distillation provides dense supervision for reasoning models via a privileged self-teacher. However, this supervision reflects local token preferences rather than the expected return of an unguided student continuation. Recent methods combine this signal with GRPO. They use GRPO for update directions and...
Fei Wu, Zhen-Rong Zhang, Jian-Shu Zhang et al.· 0 citations
The results support a compact, Fisher-conditioned privileged subspace for short-budget OPSD, with strict gains in 11 of the 12 model-checkpoint combinations and an exact tie at 4B step 25.
Yun-Meng Chen, Kun Wang, Pei-Han Li et al.· 1 citation
Modeling LLM reasoning as search over a directed acyclic graph, a unified theoretical analysis of both the dynamics of post-training---OPSD and supervised fine-tuning (SFT) in continual learning---and the impact of pre-training on subsequent performance shows that reliable continual reasoning depends on how post-traini...