Versat-AI: An ONNX-to-SoC Compiler for Model-Agnostic CGRA Edge Inference

Abstract

Edge inference on resource-constrained embedded nodes demands accelerators that are energy-efficient and compact. This paper presents Versat-AI, an open-source compiler that accepts a standard Open Neural Network Exchange (ONNX) model and generates a complete, synthesisable RISC-V System-on-Chip (SoC) with an embedded CGRA accelerator. The key innovation is applying a known hardware merge strategy to collapse structurally compatible neural network operators into a physical CGRA instance. The Versat-AI compiler also derives memory-mapped interconnects, firmware drivers, and RISC-V application software co-generated by the Py2HWSW SoC framework, eliminating the manual hardware/software co-design effort that previously tied this accelerator’s own design lineage to a single target network. The next phase of the project is to extend this same automatic derivation from sizing the operator vocabulary to sizing per-operator parallel instancing and bandwidth, following the bandwidth-matched scaling principle already demonstrated, by hand, in this accelerator’s own design lineage. The current phase of the project has succeeded in creating a sound automation flow that produces an accelerator that maps each operator onto a single physical datapath instance and occupies 8763 LUTs, 9833 flip-flops, 4 DSPs, and 202 BRAMs on a Xilinx Kintex UltraScale field-programmable gate array (FPGA)—a footprint unchanged across all evaluated models regardless of size—and draws 0.65 W (1.96 W for the complete SoC including the DDR4 controller, by Vivado post-implementation power estimation), achieving 2.3× to 9× speedup over the software-only baseline produced by the same flow on four MLPerf Tiny benchmark tasks. The paper further examines the design choices that delimit this first phase—single-precision arithmetic, a single datapath instance per operator, and the block-RAM cost of the accelerator’s streaming buffers—and sets out the path to quantised integer support and parallel operator instancing. A condensed account of the two-decade lineage of reconfigurable accelerators and open-source SoC platforms that motivated Versat-AI’s design is also given.