Book
Open access
Jul 2026
Accelerating Block Low-Rank Foundation Model Inference on Memory-Constrained GPUs
This work introduces custom Triton kernels with partial fusion and memory layout optimizations for both Monarch and BLAST, and delivers up to 3.76 × speedups and 3 × model size compression over PyTorch dense baselines using CUDA backend and compiler-level optimizations.
Pierre Abillama, Changwoo Lee, Juechu Dong et al.
· IEEE International Symposium... · 0 citations