Liminal training is introduced, an annealed KL-regularized fine-tuning method that constrains early drift from the base model and shows that KL timing matters: early regularization is more effective than late regularization, and sweeping the regularization strength reveals an empirical trade-off between task learning and trait suppression.
Abstract
Knowledge distillation can transmit unintended behavioral traits from a teacher model to a student through training data that appear semantically unrelated to those traits, a phenomenon known as subliminal learning. Although recent work has established this effect, its training dynamics and mitigation remain underexplored. We study subliminal learning in open-weight language models ranging from 1.5B to 8B parameters, covering the Qwen, Gemma, and Llama families in number-sequence and chain-of-thought settings. Rather than evaluating only final models, we track trait-related probabilities throughout fine-tuning and find that subliminal acquisition can be highly non-monotonic, with transient spikes, reversals, and trait-specific failures of transfer. We then introduce liminal training, an annealed KL-regularized fine-tuning method that constrains early drift from the base model. Across our experiments, liminal training substantially reduces subliminal trait acquisition while largely preserving task gains, outperforming paraphrasing and layer freezing as mitigation strategies. The effect also extends beyond animal preferences: in a French-language response-style experiment, liminal training suppresses language transfer while retaining much of the GSM8K improvement. Finally, we show that KL timing matters: early regularization is more effective than late regularization, and sweeping the regularization strength reveals an empirical trade-off between task learning and trait suppression.
In subliminal learning (SL), a teacher model passes on a trait to a student model by distillation on data semantically unrelated to the trait. So far, SL has been demonstrated for only a limited range of traits, including preferences for animals (e.g., owls) and malicious personas. These traits can also be elicited wit...
Jan Dubinski, A. Sztyber-Betley, Jan Betley et al.· 5 citations· ⚡1
Results show that inoculation with a learned association introduced via midtraining can shape selective generalisation, and more work is needed before this approach can become a load-bearing component in a developer's safety framework.
Kyle O'Brien, Edward James Young, Puria Radmard et al.· arXiv.org· 0 citations
This reproduction paper investigates subliminal learning, a consequence of distillation where teacher models transmit behavioral preference traits through semantically unrelated data, and supports the original paper's claims but shows they are not universal as transmission strength varies across traits and tasks.
Daan van der Weijden, Nathan Brack, Selene Baez Santamaria· Proceedings of the 9th Black...· 1 citation
Trait-direction drift is proposed and validated as a mechanism for subliminal learning: biased generation creates measurable preference gaps in teacher data, and student-recognizable gaps induce trait-aligned updates during supervised fine-tuning that accumulate into behavioral transfer.
Zhi-Xuan Liu, Zhichen Dong, Yuanjiu Fan, et al.· 1 citation
Condition-anchored functional preservation as a common design principle across the tested language-generation objectives is supported, and the implementation directly controls local denoising drift on teacher-generated completions.
This work proves that a subliminally transferred trait can be recovered from the parameter updates alone, without supervision, by exploiting a single asymmetry: the trait is expressed consistently across diverse datasets generated by the teacher, whereas task-specific and shared capabilities vary.
Joseph Egan, C. Gilligan-Lee, M. O'Riordan et al.· 1 citation
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduNov 17, 2026
By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.
MIT News · Artificial Intelligence· news.mit.eduAug 27, 2026
A new machine-learning framework aims to improve the success rate of computational protein design while moving away from results that reproduce sequences found in nature.