Discrete Global Style Tokens for Expressive Text-to-Speech Modelling

Abstract

Text-to-speech systems typically lack emotional expressiveness, and systems capable of expressive synthesis rely either on inferring emotions from text or accepting human-written labels from a small handful of emotion classes that are trained into the model. This work proposes an alternative based on learning a coarse discrete sentence-level representation for speech style, in the form of style tokens, learned in an unsupervised fashion, eliminating the need to pick a small number of emotion classes to model ahead of time. This can be used to augment an existing corpus with style labels to efficiently fine-tune existing text-to-speech models to condition speech generation on speaking style without the need for significant modification for training pipelines. The resulting representations and models can be used to perform speech style transfer using a reference clip as a source of style tokens to append to target text, achieving example-based speech style transfer without relying on predetermined style categories.