Scaling Associative Memory: A Sparse Graph Approach Versus Modern Hopfield Networks

Abstract

The modeling of associative memory increasingly draws on concepts from network science. While Modern Hopfield Networks (MHNs), often viewed as densely connected graphs with attention mechanisms, achieve high performance, their computational complexity imposes scalability challenges due to their computational cost. This paper introduces and analyzes Structural Associative Memories (SAMs), a new class of memory models based on sparse graph topologies. We provide a comparative analysis of SAMs and MHNs, examining how network structure affects storage capacity, retrieval accuracy, and efficiency. Theoretical results show that SAM capacity can scale quadratically with the number of nodes, offering clear advantages in large-scale systems. Experiments on scene recognition validate these findings and further demonstrate how mechanisms such as inhibition and parameters like context size shape performance. The results indicate that, while MHNs remain effective in dense-data settings, SAMs deliver a computationally efficient and scalable alternative for massive, sparse knowledge-graph memories. This work advances network science by presenting a scalable, graph-based architecture for cognitive modeling and quantifying the relationship between network topology and associative function.