Chaos and bifurcations in continuous language-model feedback
Feeding a pretrained language model the expected embedding of its own next-token distribution turns it into a discrete-time dynamical system with continuous states. Across Qwen2.5, OLMo-2 and Llama-3 models from 0.5B to 72B parameters, temperature and activation steering move this loop between repeating text and chaos,...