Skip to content

On Mitigation of Subliminal Learning in Large Language Models

Sep 2026 · 0 citations · 25 references
Computer Science

TL;DR

Liminal training is introduced, an annealed KL-regularized fine-tuning method that constrains early drift from the base model and shows that KL timing matters: early regularization is more effective than late regularization, and sweeping the regularization strength reveals an empirical trade-off between task learning and trait suppression.

Abstract

Knowledge distillation can transmit unintended behavioral traits from a teacher model to a student through training data that appear semantically unrelated to those traits, a phenomenon known as subliminal learning. Although recent work has established this effect, its training dynamics and mitigation remain underexplored. We study subliminal learning in open-weight language models ranging from 1.5B to 8B parameters, covering the Qwen, Gemma, and Llama families in number-sequence and chain-of-thought settings. Rather than evaluating only final models, we track trait-related probabilities throughout fine-tuning and find that subliminal acquisition can be highly non-monotonic, with transient spikes, reversals, and trait-specific failures of transfer. We then introduce liminal training, an annealed KL-regularized fine-tuning method that constrains early drift from the base model. Across our experiments, liminal training substantially reduces subliminal trait acquisition while largely preserving task gains, outperforming paraphrasing and layer freezing as mitigation strategies. The effect also extends beyond animal preferences: in a French-language response-style experiment, liminal training suppresses language transfer while retaining much of the GSM8K improvement. Finally, we show that KL timing matters: early regularization is more effective than late regularization, and sweeping the regularization strength reveals an empirical trade-off between task learning and trait suppression.

View source

Similar papers

#artificial intelligence Preprint Oct 2026

Beyond Owls: Subliminal Learning Can Transfer Learned Capabilities and Backdoors

In subliminal learning (SL), a teacher model passes on a trait to a student model by distillation on data semantically unrelated to the trait. So far, SL has been demonstrated for only a limited range of traits, including preferences for animals (e.g., owls) and malicious personas. These traits can also be elicited wit...

Jan Dubinski, A. Sztyber-Betley, Jan Betley et al. · 5 citations · ⚡1

Inoculation Midtraining with Learned Neologisms

Results show that inoculation with a learned association introduced via midtraining can shape selective generalisation, and more work is needed before this approach can become a load-bearing component in a developer's safety framework.

Kyle O'Brien, Edward James Young, Puria Radmard et al. · 0 citations
#artificial intelligence Open access Sep 2026

Reproducing and Evaluating the Generalizability of Subliminal Learning in Open-Weight Models

This reproduction paper investigates subliminal learning, a consequence of distillation where teacher models transmit behavioral preference traits through semantically unrelated data, and supports the original paper's claims but shows they are not universal as transmission strength varies across traits and tasks.

Daan van der Weijden, Nathan Brack, Selene Baez Santamaria · 1 citation
#machine learning Preprint Sep 2026

Subliminal Learning as Trait-Direction Drift: A Mechanism and Targeted Control under SFT Distillation

Trait-direction drift is proposed and validated as a mechanism for subliminal learning: biased generation creates measurable preference gaps in teacher data, and student-recognizable gaps induce trait-aligned updates during supervised fine-tuning that accumulate into behavioral transfer.

Zhi-Xuan Liu, Zhichen Dong, Yuanjiu Fan, et al. · 1 citation
#artificial intelligence Preprint Oct 2026

Defence against the dark arts of distillation: Unsupervised discovery of subliminal learning

This work proves that a subliminally transferred trait can be recovered from the parameter updates alone, without supervision, by exploiting a single asymmetry: the trait is expressed consistently across diverse datasets generated by the teacher, whereas task-specific and shared capabilities vary.

Joseph Egan, C. Gilligan-Lee, M. O'Riordan et al. · 1 citation

Related blog posts

MIT News · Artificial Intelligence Nov 17, 2026

Learning the language of machine learning

By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.

MIT News · Artificial Intelligence Aug 27, 2026

Looking beyond natural sequences

A new machine-learning framework aims to improve the success rate of computational protein design while moving away from results that reproduce sequences found in nature.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.