Skip to content

Author

Masahiro Tanaka

3 papers indexed here

We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.

Not the right person? Other researchers publish under this name.

Nov 2026

Pipelining the Memory Hierarchy for Ultra-Long-Context Language Model Training

FPDT democratizes long sequence training, making it accessible on mobile and commodity hardware, such as high-end local machines, and trains contexts up to 16x longer than prior sequence-parallel baselines on the same hardware.

Jing-Han Yao, Sam Ade Jacobs, Masahiro Tanaka et al. · 0 citations
Open access May 2025

ZenFlow: Enabling Stall-Free Offloading for LLM Training

ZenFlow is presented, a new offloaded training framework that decouples updates between GPU and CPU by prioritizing important gradients and achieves up to 5× end-to-end speedup, 2× lower PCIe traffic, 85% less GPU stalls, and up to 72% lower cloud GPU cost, all while preserving accuracy.

Ting-Feng Lan, Yu-Sen Wu, Bin Ma et al. · 8 citations · ⚡3
Open access Sep 2026

ZenFlow: Enabling Stall-Free Offloading for LLM Training

Fine-tuning large language models (LLMs) often exceeds GPU memory limits, prompting systems to offload model states to CPU memory. However, existing offloaded training frameworks like ZeRO-Offload treat all parameters equally and update the full model on the CPU, causing severe GPU stalls, where fast, expensive GPUs si...

Ting-Feng Lan, Yu-Sen Wu, Bin Ma et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.