Enhancing BNN storage and performance requirements via efficient quantization and variable encoding
Binary Neural Networks (BNNs) perform competitively in practical image classification tasks. In particular, their accuracy is similar to that obtained by full-precision models tailored to edge devices. BNNs are amenable for devices with constrained resources, since they employ just one bit to store inputs and weights; hence, their storage requirements are low. Moreover, BNN computations utilize XNOR operations, which are implemented efficiently using simple hardware structures. However, supporting BNNs efficiently on edge devices is challenging since they perform many memory accesses to load weights, hindering their benefits. In BNNs, weights and inputs are stored using one bit, and several of them are packed together as a sequence of bits. In this work, the number of unique sequences representing a set of weights is observed to be low. In addition, a small group of unique sequences is seen to be used more frequently than others. Accordingly, this observation is exploited by using variable length encoding to represent the most commonly used sequences and decompressing them during inference. Commonly, BNNs employ floating-point values to store the weights for the output layer, aiming to obtain competitive accuracy. This further magnifies the storage requirements and significantly affects performance. In this work, it is observed that using traditional quantization schemes to quantize these weights negatively affects network accuracy. Then, it is shown that one can use a small number of bits to store most of the values in the output layer without negatively impacting accuracy by carefully quantizing the weights that fall outside a given range. Finally, an efficient quantization scheme for the output layer is proposed. A previously proposed FPGA accelerator is extended with the required hardware to implement the proposed schemes. The proposal is evaluated on a Pynq Z1 device. Our techniques reduce memory requirement by 1.5 × and improve performance by 2.2 × on average.