Cost and Energy-Aware Evaluation of YOLO Architectures for Hierarchical Industrial AIoT

Abstract

Industrial AIoT deployments require real-time person detection under strict latency, bandwidth, and cost constraints. Hierarchical inference architectures (Device-Edge-Fog) are commonly adopted under the implicit assumption that highertier nodes provide higher accuracy at the expense of latency, while lower-tier accelerators prioritize speed over precision. In this Work-in-Progress, we empirically reassess this assumption by benchmarking twelve YOLO variants (YOLOv5, YOLOv8, YOLO11; sizes n/s/m/l) under FP16 and INT8 TensorRT optimization on an NVIDIA Jetson AGX Orin platform and comparing them with a Raspberry Pi 5 + Hailo-8 edge configuration. Using two real industrial video streams, we evaluate inference latency and detection accuracy (mAP50-95) and analyze the resulting accuracy-latency Pareto frontier. Results show that the Jetson platform dominates the accuracy-latency trade-off across all comparable model sizes. However, when introducing a cost-normalized efficiency metric, the Hailo-based configuration achieves up to $1.7\times$ higher performance-per-euro efficiency despite lower absolute performance. These findings challenge simplified design assumptions in hierarchical AI systems and demonstrate that hardware optimality depends on the abstraction level of analysis (node-level vs. systemlevel). The presented evaluation framework provides quantitative foundations for future adaptive offloading policies in industrial Edge AI deployments.