Albert Gatt

Language Re-generation: An investigation into information locality effects on reconstruction

This work combines learnability-based approaches with a reconstruction framework, fine-tuning GPT-2 models pre-trained on impossible languages to reconstruct natural English from three perturbation types and shows that the recovered structures exhibit shorter dependency lengths, mirroring the locality preference observed in unconstrained language model generation.