Nov 2026
Pipelining the Memory Hierarchy for Ultra-Long-Context Language Model Training
FPDT democratizes long sequence training, making it accessible on mobile and commodity hardware, such as high-end local machines, and trains contexts up to 16x longer than prior sequence-parallel baselines on the same hardware.
Jing-Han Yao, Sam Ade Jacobs, Masahiro Tanaka et al.
· GetMobile Mobile Computing a... · 0 citations