A Streamwise GAN Vocoder for Wideband Speech Coding at Very Low Bit Rate

Abstract: Recently, GAN vocoders have seen rapid progress in speech synthesis, starting to outperform autoregressive models in perceptual quality with much higher generation speed. However, autoregressive vocoders are still the common choice for neural generation of speech signals coded at very low bit rates. In this paper, we present a GAN vocoder which is able to generate wideband speech waveforms from parameters coded at 1.6 kbit/s. The proposed model is a modified version of the StyleMelGAN vocoder that can run in frame-by-frame manner, making it suitable for streaming applications. The experimental results show that the proposed model significantly outperforms prior autoregressive vocoders like LPCNet for very low bit rate speech coding, with computational complexity of about 5 GMACs, providing a new state of the art in this domain. Moreover, this streamwise adversarial vocoder delivers quality competitive to advanced speech codecs such as EVS at 5.9 kbit/s on clean speech, which motivates further usage of feed-forward fully-convolutional models for low bit rate speech coding.

Preprint: arxiv, accepted to WASPAA 2021

- For all samples in this demo:

• Both SSMGAN (our proposed model) and LPCNet are trained on the full training set of the VCTK corpus provided under Creative Commons License.
• LPCNet is built and trained based on the official implementation.
• The Lyra codec is obtained from the official repository, with version 0.0.1 and commit 9358429 which is the latest update at the time of releasing this demo.

- Conditions of Use.

---

Seen VCTK Speakers:

Original	SSMGAN 1.6kbit/s (ours)	LPCNet 1.6kbit/s	Lyra 3kbit/s	EVS 5.9kbit/s	AMR-WB 6.6kbit/s	Speex 4kbit/s

---

Unseen VCTK Speakers:

Original	SSMGAN 1.6kbit/s (ours)	LPCNet 1.6kbit/s	Lyra 3kbit/s	EVS 5.9kbit/s	AMR-WB 6.6kbit/s	Speex 4kbit/s

---

Unseen CMU ARCTIC Speakers

• Because the VCTK corpus has only 2 unseen speakers for testing, we resort to the CMU ARCTIC Database to obtain more unseen speakers.
• It provides clean speech signals with similar recording quality to VCTK.
• © Carnegie Mellon University, 2003, All Rights Reserved, license.

Original	SSMGAN 1.6kbit/s (ours)	LPCNet 1.6kbit/s	Lyra 3kbit/s	EVS 5.9kbit/s	AMR-WB 6.6kbit/s	Speex 4kbit/s

---

Conditions of Use:

• Fraunhofer IIS generated this sound material based on material that is freely available on The Centre for Speech Technology Research (CSTR) at Edinburgh University for VCTK and The Language Technologies Institute at Carnegie Mellon University for CMU ARCTIC.
• The content has been processed using generally accepted rules of technology as well as scientific care, but not actual attainment of any expected feature.
• With the exception of willful intent or gross negligence, Fraunhofer IIS shall not be liable that Open Source software or other third party software is free from any error or claim or its fitness for a particular purpose, even if included within the Sound Material.
• The Sound Material shall only be used for testing and appreciating speech synthesis techniques and shall not be copied, publicly transmitted, distributed, lent and modified for any other reason.
• No representation or warranties are made or implied regarding the accuracy, non-infringement, fitness for a particular purpose of Sound Material.
• Unless the conditions shown above are agreed by the recipient, no permission is granted for any use including copy, public transmission, distribution, lend, and modification.
• Copyright and Permission notice shall be duplicated whenever Sound Material are copied, distributed or publicly transmitted.
• The Sound material cannot be distributed with charge. ^[return]