Microscaling (MX) is now the standard for low-bit large language model (LLM) inference. Its 4-bit form MXFP4 still loses substantial accuracy, because existing MX formats fix either the element format or the precision-recovery scheme across blocks, and thus capture only limited quantization heterogeneity. Quantization heterogeneity appears at two levels: 1) across blocks, the preferred element format and precision-recovery scheme vary; 2) across operands, weights and activations require different encoding. We introduce AdaMX (Adaptive Microscaling), a heterogeneity-aware format and accelerator. It selects the precision-recovery scheme per block and the representation per operand, at no increase in equivalent bit width (EBW). One design covers two block sizes, giving a higher-accuracy operating point and a lower-EBW operating point that saves storage. We implement a 22nm FD-SOI AI accelerator prototype with the proposed decoder, computing unit, and quantization logic. Against an otherwise identical MXFP4 accelerator with FP4-only multipliers, AdaMX adds about 1% system energy. At the lower-EBW point, AdaMX stays more accurate than the baseline while lowering both memory footprint and energy. Across LLMs from 3B to 70B, AdaMX removes 83% of the MXFP4 accuracy loss on commonsense and 82% on MMLU, and 43% and 27% of the NVFP4 loss. AdaMX also generalizes to multimodal models. On Gemma-4 12B, it leads MXFP4 on all four vision-language benchmarks and keeps up to 96% of FP16 accuracy.
Junyi Luo, Xin Jiang, Tai-Hao Wen et al.· 0 citations
Edge devices capable of running artificial intelligence (AI) applications have seen a surge in demand for energy-efficient and high-throughput computation. In this study, a 22-nm edge–AI processor, incorporating an accelerator with error-free Booth-value-confined (BVC) multiprecision (MP) multiplier and near-memory computing (NMC), is introduced to accelerate neural networks (NNs). It has the following three major features. First, a BVC MP multiplier based on radix-8 Booth (R8B) is introduced to reduce computation complexity by prohibiting the “±3” cases and support error-free training on GPU without accuracy loss originating from the mismatch between training and deployment. A PE is built based on this multiplier for parallel computation with 82% power reduction and 70% area reduction. Second, the proposed NMC-friendly data flow supports efficient data reuse and hence reduces off-chip memory traffic. The data flow supports data reuse of up to 16 times, matching the number of PEs and enabling regular read and write patterns. Third, a hardware-aware layer-wise model deployment approach is proposed with a memory space contiguity-aware (MSCA) model reshape strategy, and a hardware-aware NN splitting and scheduling algorithm. The proposed MSCA strategy maximizes burst access, and the proposed algorithm achieves efficient computation with high data reuse and low memory access. This deployment approach can achieve a reduction in memory access latency of 16.6%–32.0%. Measurements on a 22-nm test chip demonstrate a peak power efficiency of 33.98 TOPS/W under synthetic full-PE-utilization conditions, while achieving 12.92–29.11 TOPS/W for end-to-end NN inference on DarkNet19, VGG16, ViT-Tiny, and ResNet34.
Quan Cheng, Longyang Lin, Mingqiang Huang et al.· IEEE Transactions on Very La...· 0 citations