Parts-of-Speech as Emergent Categories in SAE Latent Space
Sparse AutoEncoders (SAEs) offer a promising way to inspect language model representations, but it is still unclear what kind of linguistic structure their latents expose. We use part-of-speech (PoS) categories as a controlled test case to study whether morpho-syntactic information is encoded by individual latents or b...
Alessandro Bondielli, Lucia C. Passaro, Serena Auriemma et al.
· 0 citations