Skip to content
Book Open access

Align-TCN: A Hardware-Aligned Streaming Architecture for On-Device Multimedia Concurrency

Nov 2026 · Proceedings of the 34th ACM International Conference on Multimedia · 0 citations · 34 references

Abstract

Speech enhancement is an always-on component in mobile multimedia applications, where it must operate under stringent latency budgets alongside other computation-intensive vision tasks, such as real-time HD video calling and concurrent computational photography. Although existing lightweight speech enhancement models deliver strong acoustic performance with minimal parameter counts, their hard real-time streaming deployment on on-device NPU incurs substantial inference latency, which reduces the scheduling budget for concurrent tasks and degrades the overall real-time responsiveness of multimedia systems. To address this issue, we present Align-TCN, a hardware-aligned fully convolutional streaming architecture for on-device deployment. The proposed architecture is co-designed from both representation and computation perspectives, by constructing an aligned spectral representation that better matches NPU execution characteristics and by introducing time-frequency 2D causal convolutions for efficient parallel modeling of streaming speech features. In this way, Align-TCN achieves a better balance between deployment efficiency and enhancement performance under stringent real-time constraints, with a per-frame inference latency of only 1 ms on edge NPU. Extensive experiments on both standalone deployment and end-to-end concurrent multimedia pipelines validate the effectiveness of the proposed design. Compared to existing lightweight baselines, Align-TCN maintains a similar parameter budget while substantially reducing per-frame inference latency on edge NPU. It further improves system-level real-time performance in concurrent multimedia pipelines with 60 FPS guaranteed for concurrent tasks, while preserving competitive acoustic quality.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.