In-Memory Computing Architectures for Accelerating Large-Scale Models: Principles, Devices, and System Challenges

Abstract

Memory bandwidth is the binding constraint for large-model inference today. In-memory computing (IMC) addresses this limitation. This paper reviews where the technology stands. It compares analog and digital IMC implementations and find that analog arrays currently lead on energy efficiency, while digital designs maintain a substantial edge in precision and programming flexibility. Then it discusses three device candidates: Resistive Random-Access Memory (ReRAM), Ferroelectric Field-Effect Transistor (FeFET), and Spin-Transfer-Torque Magnetic Random-Access Memory (STT-MRAM). ReRAM offers the highest density but suffers from stochastic filament growth and conductance drift. FeFETs integrate cleanly with CMOS but are limited by ferroelectric switching variability. STT-MRAM is the most durable and stable, yet its modest tunnel-magnetoresistance ratio restricts analog precision. Finally, three limitations are pointed out: device variation under realistic workloads, peripheral-circuit overhead that can erase array-level energy gains, and scalable multi-chip orchestration for trillion-parameter models. IMC will succeed or fail based on cross-layer co-design—not on any single device breakthrough. Materials, circuits, compilers, and algorithms must advance together, and the compiler layer remains underdeveloped relative to device research. This paper provides development ideas for higher performance for IMC.