Codebook refinement transformer: post-retrieval correction for neural codec voice conversion
We propose the Codebook Refinement Transformer (CRT), a non-autoregressive model that improves retrieval-based voice conversion on discrete neural audio codec tokens. CRT operates on the Mimi codec’s hierarchical residual vector quantized (RVQ) token space using a Conformer backbone with Feature-wise Linear Modulation...