Mohammed Suhail B Nadaf

Emergent Misalignment Recruits a Pre-existing Persona Subspace

It is found that narrow fine-tuning recruits a persona structure that is present in the model before the fine-tune exists, and that broad misalignment on questions unrelated to the training data is more broad than mechanical weight superposition and matched diversity jointly account for.