Xiaowei Li

HydraPIM: A Heterogeneous PIM Architecture for Efficient Attention in Long-Context LLMs

The growing demand for long-context LLM inference has exposed a critical bandwidth–capacity trade-off in memory systems, rendering single-tier PIM architectures ineffective. HBM-PIMs offer high bandwidth but limited capacity, while DIMM-PIMs provide scalability at the cost of lower bandwidth; neither satisfies the thro...