Information-Bottleneck Disentanglement for Controllable Singing Voice Conversion

Abstract

Singing voice conversion is a challenging task in audio signal processing that aims to transform the perceived identity of a source singer into that of a target singer while preserving the underlying linguistic content and musical melody. The inherent complexity of singing voices, which intertwine phonetic structures, pitch contours, and unique vocal tract characteristics, makes the complete disentanglement of these features notoriously difficult. This paper presents a novel approach to controllable singing voice conversion through the application of the information-bottleneck principle. By imposing an information-bottleneck constraint on the latent representation of the source audio, our proposed methodology successfully restricts the flow of identity-specific information, thereby forcing the acoustic encoder to retain only the essential linguistic and melodic features. The subsequent injection of target speaker embeddings and explicit pitch contours enables highly controllable and naturalistic voice conversion. We provide a comprehensive theoretical analysis of the information-bottleneck framework adapted for sequential audio data and outline a robust architecture designed to exploit this principle. Extensive objective and subjective evaluations demonstrate that the proposed model significantly outperforms existing baseline systems in terms of both voice similarity and audio quality. Furthermore, the explicit disentanglement allows for unprecedented post-synthesis control over pitch scaling, vibrato manipulation, and rhythm adjustment, opening new avenues for creative vocal production.