CNN Architecture Optimization for Multiplierless Inference
TL;DR
This work addresses the gap in CNN architectures with subsequent In-Memory Computing implementation in mind, using an analytical resource–cost model for multiplierless integer inference, and constrains candidate CNN architectures toward integer computations that can be expressed through memory look-ups, additions, and shifts, rather than conventional multiply–accumulate operations.
Abstract
Many CNN architectures are primarily optimized for classification accuracy and software-level metrics, but they do not necessarily account for resource-constrained integer inference. This work addresses this gap by optimizing CNN architectures with subsequent In-Memory Computing (IMC) implementation in mind, using an analytical resource–cost model for multiplierless integer inference. Motivated by the need to reduce data movement and avoid explicit multipliers, the proposed methodology constrains candidate CNN architectures toward integer computations that can be expressed through memory look-ups, additions, and shifts, rather than conventional multiply–accumulate operations. Distributed Arithmetic provides the computational basis for this multiplierless formulation, while the proposed HWCost model is used together with validation performance and validation efficiency to rank and select candidate architectures. Candidate architectures are generated by combining six redesign techniques: Global Average Pooling (GAP) to reduce classifier complexity, Quantization-Aware Training (QAT) to optimize weight precision, reduction in convolution filter size, selection of the downsampling strategy, power-of-two constraints on the feature map size before GAP, and training oriented toward integer-only inference. We evaluate the combined effect of these techniques on MNIST, FashionMNIST, CIFAR-10, and PneumoniaMNIST. The results show that, across MNIST and FashionMNIST, the selected compact models reduce the estimated HWCost by up to 2.52× with approximately one percentage point or less degradation in test accuracy. On CIFAR-10, the lowest-cost candidate reduces the estimated HWCost by 3.94×, relative to the highest-accuracy candidate, at the cost of a larger accuracy reduction. On PneumoniaMNIST, the efficiency-oriented candidate achieves comparable independent-test-balanced accuracy to the highest-validation-balanced accuracy candidate while requiring substantially lower estimated HWCost.