M. Tsatsos

Linear Probes Transfer from Base to Distilled Language Models despite Rotation

A probe trained on a base model can be reused, in these families, after distillation and after instruction tuning; how much it reads beyond textual cues depends on the test set.