Skip to content
Log in

Author

Yi Han

View on Semantic Scholar

2 papers indexed here

We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.

Not the right person? Other researchers publish under this name.

Sep 2026

HydraPIM: A Heterogeneous PIM Architecture for Efficient Attention in Long-Context LLMs

The growing demand for long-context LLM inference has exposed a critical bandwidth–capacity trade-off in memory systems, rendering single-tier PIM architectures ineffective. HBM-PIMs offer high bandwidth but limited capacity, while DIMM-PIMs provide scalability at the cost of lower bandwidth; neither satisfies the throughput-latency requirements of long-context serving workloads for LLMs. To address this, we propose <bold>HydraPIM</bold>, a heterogeneous multi-tier PIM architecture that decomposes attention computation across HBM-PIM and DIMM-PIM tiers by exploiting the <bold>inherent sparsity</bold> of attention. HydraPIM introduces <bold>HydraAttention</bold>, a tiled attention mechanism with hierarchical reduction, enabling efficient cross-tier execution through lightweight on-chip reduction units. To maintain load balance under dynamic access patterns of sparse attention, HydraPIM features an <bold>importance-aware KV migration</bold> mechanism that monitors token relevance and relocates hot tokens to high-bandwidth tiers at runtime. This software-hardware co-design helps improve the utilization of both bandwidth and capacity. Evaluations show that HydraPIM achieves 1.66<inline-formula><tex-math notation="LaTeX">$\boldsymbol{\times}$</tex-math><alternatives><mml:math><mml:mo mathvariant="bold">×</mml:mo></mml:math><inline-graphic xlink:href="wang-ieq1-3710733.gif"/></alternatives></inline-formula> higher throughput than HBM-based NPU-PIM and 1.96<inline-formula><tex-math notation="LaTeX">$\boldsymbol{\times}$</tex-math><alternatives><mml:math><mml:mo mathvariant="bold">×</mml:mo></mml:math><inline-graphic xlink:href="wang-ieq2-3710733.gif"/></alternatives></inline-formula> lower latency than DIMM-based NPU-PIM across diverse long-context workloads.

Shixin Zhao, Lian Liu, Xiangwen An et al. · 0 citations
Preprint Aug 2026

Rethinking Unified Memory for NPU-PIM Systems: Dual-View Memory for Dynamic Inference of LLM

Heterogeneous architectures that combine neural processing unit (NPU) and processing-in-memory (PIM) are increasingly adopted to accelerate LLM inference. Prior work focuses on building a unified memory that allows NPUs and PIM to share data without duplication. However, these designs implicitly assume that each tensor is bound to a fixed execution device, and therefore rely on static, device-biased data mappings. We observe that this assumption does not hold in modern LLM workloads. Due to phase changes (e.g., prefill vs. decode) and dynamic behaviors such as MoE routing, the optimal execution device for the same tensor can change at runtime. Under such dynamic execution, device-biased mappings become mismatched to access patterns, leading to substantial bandwidth underutilization and performance loss. This paper presents PFM (PIM-as-Flexible-Memory), a dual-view memory system that decouples physical data layout from accessor-visible logical views. PFM stores data in a jointly optimized physical layout and exposes different logical interpretations to NPUs and PIM, enabling efficient access across devices without data duplication or relayout. We further design accessor-aware address translation and runtime scheduling mechanisms to support dynamic execution when LLM workloads fluctuate and the optimal execution device dynamically changes. Our evaluation across LLMs shows that PFM improves end-to-end throughput by up to 2.32$\times$, demonstrating its effectiveness and broad applicability as a unified memory management solution for NPU-PIM systems.

Shixin Zhao, Lian Liu, Tian Han et al. · 0 citations