A theory of platonic representations in language models
Representations of translated sentences are similar in the inner layers of multilingual language models -- an observation connected to the platonic representation hypothesis, yet unexplained theoretically. We provide an explanation based on the assumption that data have a hidden hierarchical structure whose abstract le...
Darshil Doshi, Wen-Jie Zhou, C. Wegner et al.
· 0 citations