Location-Aware Mixture-of-Experts Framework for Adaptive Speech Enhancement

Abstract

Speech enhancement models often struggle to adapt to heterogeneous and ambiguous noise conditions. This paper proposes a location-aware mixture-of-experts framework that jointly uses acoustic observations and categorical location context for adaptive speech enhancement. A gating network estimates continuous expert weights from acoustic and location embeddings, and the enhanced magnitude spectrum is obtained by softly aggregating multiple expert estimates before waveform reconstruction. The framework is trained end-to-end without predefined expert assignments or acoustic-scene supervision. Experiments using LibriSpeech and the TAU Urban Acoustic Scenes 2022 Mobile dataset show that the proposed method consistently outperforms conventional and recent speech enhancement approaches across different SNR levels and acoustic environments. Additional analyses demonstrate robustness to mixed noise and location uncertainty and show that meaningful expert specialization emerges during training. These results confirm that location information can serve as a useful contextual cue for robust speech enhancement when combined with acoustic evidence through soft expert aggregation.