Binarized Wake-Up of Conversational Agents on an Industry-Grade High-Performance Microcontroller

Abstract

Quantization is an effective technique for field-deployment of neural networks (NNs) within applications on resource-limited devices. A representative case is the wake-up of conversational agents, where a camera-bearing edge node continuously monitors the environment to detect the visual trigger through which a user starts the interaction with the agent. While the state of the art on the WakeVision person-detection benchmark spans floating-point and up to 4-bit quantized models, binary quantization has not been addressed in the literature yet. Binarization frequently allows the implementation of high resource-efficiency solutions; however, it is challenging to implement since, differently from the other quantization types, it requires specific binary operations, that are not provided by standard off-the-shelf machine learning libraries for microcontrollers. To explore the issue, we study two binary deployments on a STM32H7B3I-DK high-end microcontroller. First, we use NAS-BNN, a framework dedicated to optimizing binary NNs, to discover and deploy a binary convolutional NN via the Larq Compute Engine; to fit the memory budget, we resort to a 16-tile spatial input-streaming scheme. Second, we binarize the official WakeVision 4-bit Samy model using the CBin-NN inference engine, achieving the best overall trade-off in terms of latency, memory footprint, and energy consumption. This highlights the effectiveness of tightly coupling hardware-aware model design with deployment via a dedicated, binary-optimized runtime. These findings set the reference for minimal footprint deployment of NN solutions for the visual wake-up of conversational agents and indicate promising directions for future research.