MAGSAT: Memory-Access-Guided Fusion and Shape-Adaptive Auto-Tuning for Dynamic Tensor Programs

Abstract

Dynamic tensor programs are increasingly common in deep learning. Existing libraries and compilers provide or generate highly optimized kernels, but their decisions are often tied to fixed operator patterns or concrete shapes. Dynamic-shape systems reduce tuning overhead, yet they typically optimize individual operators and leave graph-level fusion opportunities underexplored. We propose MAGSAT, a unified framework that combines shape-invariant operator fusion with shape-adaptive parameter selection for dynamic workloads. MAGSAT analyzes memory-access semantics to identify fusion regions, maps them to parameterized tiled templates, and selects kernel configurations with a lightweight trait-driven analytical model. Extensive experiments demonstrate that MAGSAT achieves average speedups of 4.65 × for fused operators and 2.74 × for end-to-end inference over the state-of-the-art baseline.