World Modeling in Transformers
TaxiGPT, a transformer trained on random walks through Manhattan whose failures have been interpreted as evidence of an incoherent internal map, is demonstrated, through mechanistic analysis and causal interventions, that the model represents intersections and streets, tracks its position, and uses a goal compass to na...