Open access
Jun 2026
When transformers learn "impossible" languages, what do they learn?
Using GPT-2 style models trained on perturbed"impossible"variants of English, sensitivity to grammaticality is measured using BLiMP minimal pairs, finding that model performance exhibits only gradual degradation, mediated by the language's information locality.
Ram Janarthan, Coleman Haley, Sharon Goldwater
· Conference on Computational... · 0 citations