Persistent Sparse Autoencoders: Learning Feature-Specific Timescales in Language Model Representations
This work introduces Persistent Sparse Autoencoders (Persistent SAEs), an extension of standard SAEs that learns a persistence coefficient for each feature, allowing the model to learn feature-specific timescales from reconstruction alone.
Haoyang Luo, M. Zarlenga, M. Jamnik
· 0 citations