Deep Learning Models Also Recall Features
TL;DR
This paper argues that factual recall points to something broader: a general kind of operation in deep learning models, which is called feature recall, and defines it, shows it applies across architectures, and contrasts it with the established paradigm of feature combination.
Abstract
Recent work in mechanistic interpretability has studied how large language models recall facts stored in their weights. This paper argues that factual recall points to something broader: a general kind of operation in deep learning models, which I call feature recall. The core observation is that a linear projection can be read as retrieving stored information scaled by input activations. I define feature recall, show it applies across architectures, and contrast it with the established paradigm of feature combination. I also consider how cases of feature recall might be mechanistically identified. The account gives philosophers a new conceptual tool for understanding deep learning, and points to empirical directions for mechanistic interpretability research.