Mixture-of-Experts (MoE) have become a mainstream way to scale Large Language Models (LLMs), because each token activates only a small subset of experts, reducing inference cost. However, MoE decode remains highly memory-bound and exhibits strongly skewed, time-varying expert loads. 3D-stacked DRAM near-memory processi...
Yu-Dong Pan, Lian Liu, Shi-Xin Zhao et al.· 0 citations
The growing demand for long-context LLM inference has exposed a critical bandwidth–capacity trade-off in memory systems, rendering single-tier PIM architectures ineffective. HBM-PIMs offer high bandwidth but limited capacity, while DIMM-PIMs provide scalability at the cost of lower bandwidth; neither satisfies the thro...
Shixin Zhao, Lian Liu, Xiangwen An et al.· IEEE transactions on compute...· 0 citations
PFM (PIM-as-Flexible-Memory), a dual-view memory system that decouples physical data layout from accessor-visible logical views, is presented, demonstrating its effectiveness and broad applicability as a unified memory management solution for NPU-PIM systems.
Shixin Zhao, Lian Liu, Tian Han et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.