Persistent Sparse Autoencoders: Learning Feature-Specific Timescales in Language Model Representations
This work introduces Persistent Sparse Autoencoders (Persistent SAEs), an extension of standard SAEs that learns a persistence coefficient for each feature, allowing the model to learn feature-specific timescales from reconstruction alone.