Global Divergence, Local Convergence: Representation Geometry in SSMs and Transformers
A multi-scale analysis of representations in transformers, SSMs, and hybrid architecture suggests that while transformers and SSMs induce different usage of latent space, they display a striking functional convergence at the level of local semantic manifolds.
Amit Ben-Artzy, Roy Schwartz
· 0 citations