Spontaneous Context Restoration: How Language Models Recover from Corrupted Inputs
Moderate-corruption finetuning increases corruption tolerance while simultaneously reducing displacement-normalized linearization error, associating improved robustness with a more nearly linear response to corruption.