Neural audio codecs compress waveforms into compact discrete tokens that underpin speech language models, real-time communication, and large-scale audio storage. Almost every dominant design, including residual vector quantization, finite scalar quantization, and single-codebook variants, follows the VQ-VAE template by...
This work introduces Iterative Latent Representation Refinement (ILRR), an efficient framework for steering DLMs using a reference text as a high-level semantic blueprint and introduces Spatially Modulated Steering, an extension that enables long-form generation to be guided by shorter references by regulating intensit...
Eden Avrahami, Eliya Nachmani· arXiv.org· 5 citations
SIEDD, a discrete diffusion framework for text-guided speech inpainting and editing over hierarchical codec tokens, is introduced and results demonstrate that explicitly modeling the codec hierarchy substantially improves context-preserving speech reconstruction and editing.
Iftach Shoham, Tali Dror, Oren Gal et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.