A Nonvolatile AI-Edge Processor With Lossless-Compressed-Computing STT-MRAM Near-Memory-Compute Macro Using Dynamic Floating-/Fixed-Point Accumulation
Abstract
Nonvolatile AI-edge processors based on near-memory-compute (nvNMC) enable energy-efficient multiply-and-accumulate (MAC) operations with short wakeup latency for edge inference operations. Lossless compression is required for floating-point (FP) neural network (NN) models under on-chip memory capacity constraints; however, this imposes several challenges: 1) data decompression overhead due to lossless FP weight encoding; 2) a read-yield–energy tradeoff due to near-far effects in large nonvolatile memory arrays; 3) a MAC-accuracy–energy tradeoff in FP accumulation within the multi-level adder tree; and 4) inefficient layer fusion due to feature-map (FM)–weight mapping in nvNMC. This article addresses these challenges by presenting a nonvolatile AI-edge processor featuring four system–circuit co-design schemes: 1) delta-exponent lossless compression computation (DELC ${}^{2}$ ); 2) near-far-aware bias-and-mapping (NFABM); 3) dynamic floating-/fixed-point (FXP) accumulation (DF2PA); and 4) nvNMC-friendly hybrid-layer-fusion weight-mapping (CIM-HLF-WM). A test chip fabricated using foundry-provided 22-nm spin-transfer-torque magnetic random access memory (STT-MRAM) achieved an energy efficiency of 24.6 TFLOPS/W and a wakeup-to-response latency of 428.58 $\mu $ s under BF16 input and weight, verifying its effectiveness for capacity- and energy-constrained AI-edge applications.