Natural TTS Synthesis By Conditioning Wavenet On Mel Spectrogram Predictions | Awesome LLM Papers Contribute to Awesome LLM Papers

Awesome LLM Papers

A curated collection of research papers on Large Language Models (LLMs), transformers, prompting, fine-tuning, and multi-modal systems. Maintained by Sean Moran.

Contact Sean Moran about this survey or website. Made with Jekyll and Hyde.

Natural TTS Synthesis By Conditioning Wavenet On Mel Spectrogram Predictions

Jonathan Shen, Ruoming Pang, Ron J. Weiss, Mike Schuster, Navdeep Jaitly, Zongheng Yang, Zhifeng Chen, Yu Zhang, Yuxuan Wang, Rj Skerry-Ryan, Rif A. Saurous, Yannis Agiomyrgiannakis, Yonghui Wu . 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2018 – 2350 citations

[Paper]   Search on Google Scholar   Search on Semantic Scholar
ICASSP Model Architecture

This paper describes Tacotron 2, a neural network architecture for speech synthesis directly from text. The system is composed of a recurrent sequence-to-sequence feature prediction network that maps character embeddings to mel-scale spectrograms, followed by a modified WaveNet model acting as a vocoder to synthesize timedomain waveforms from those spectrograms. Our model achieves a mean opinion score (MOS) of (4.53) comparable to a MOS of (4.58) for professionally recorded speech. To validate our design choices, we present ablation studies of key components of our system and evaluate the impact of using mel spectrograms as the input to WaveNet instead of linguistic, duration, and (F_0) features. We further demonstrate that using a compact acoustic intermediate representation enables significant simplification of the WaveNet architecture.

Similar Work