Don't Read Everything: A Curvature-Conditioned Query for Linear Attention
Curvature-Conditioned Query modifies only the read step and is composable with any linear-attention backbone, and improves perplexity, zero-shot downstream accuracy, S-NIAH retrieval at and beyond the training context, length-extrapolation perplexity from 4K to 20K, and LongBench accuracy.