DV-SSL: Direct Vector-Based Real-Time 3D Sound Source Localization for Humanoid HRI

Abstract

Accurate and low-latency sound source direction estimation is essential for natural human-robot interaction. However, existing deep learning-based direction-of-arrival (DoA) estimation studies have primarily focused on improving estimation accuracy, while validation of real-time operation and spatial robustness in practical interactive environments remains limited. In this paper, we propose a lightweight real-time 3D DoA estimation framework, termed Direct-Vector Sound Source Localization (DV-SSL). The proposed model directly regresses a Cartesian direction vector, thereby removing grid-based post-processing and reducing computational cost. In addition, the architecture incorporates a stateful GRU-based streaming structure, frequency pooling, and frequency attention pooling to improve efficiency and robustness in continuous inference scenarios. The proposed system was trained on simulation data and evaluated under various azimuth and elevation conditions in both simulation and a humanoid upper-body torso robot testbed. Experimental results show that the proposed method achieves stable real-time operation and robust 3D sound source direction estimation performance, demonstrating its applicability to humanoid robot based human-robot interaction systems.