I. Piontkovskaia

First-Order Predictable but Pairwise Fragile: Local Task Adaptation in Trained Transformers

Pairwise structure proves to be far more fragile: on over a third of the measured (model, task pair) combinations, two-update order sensitivity sets in strictly inside that window; task-gradient subspaces rotate within tens of steps; additivity under the fixed activation probe fails at full task-vector scale on several...