MP3: Mixed-Precision Pipeline Parallelism Framework for Heterogeneous Edge Devices

Abstract

Owing to the inherent advantages of edge computing in latency, privacy, and always-on availability, deep neural network (DNN) tasks are progressively shifting to edge devices. However, limited edge resources pose significant challenges for deploying large DNNs. Pipeline parallelism (PP) and post-training quantization (PTQ) are essential techniques for addressing these resource constraints, and mainstream frameworks (e.g., DeepSpeed, PyTorch) encapsulate them independently. Nonetheless, this decoupled treatment inevitably traps the optimization process into local optima and results in inferior throughput-accuracy tradeoffs in heterogeneous edge environments, due to the natural circular dependency between the PP and PTQ optimization processes. To this end, we propose MP3, a novel framework for joint PP-PTQ optimization in heterogeneous edge environments. MP3 integrates three core components: 1) task-device co-analyzer (TDC) utilizes a fine-grained modeling method to estimate computation and communication latencies precisely; 2) information-fused accuracy proxy (FAP) characterizes quantized DNN accuracy through a unified proxy combining intralayer and interlayer information, with a data-driven fusion function search strategy; and 3) pipeline quantization co-optimizer (PQC), leveraging latency estimates from TDC and accuracy characterizations from FAP, navigates the joint optimization space effectively by formulating it as a constraint programming (CP) problem. A comprehensive experimental evaluation on an edge testbed confirms the effectiveness of MP3, demonstrating that it lies on a superior Pareto curve (achieving up to $3.2\times $ throughput speedup with accuracy drop within 1%) and exhibits robust adaptability across heterogeneous devices.