mms-tts-eng EU AI Act Compliance Profile

license: cc-by-nc-4.0 tags:

• mms
• vits pipeline_tag: text-to-speech

Massively Multilingual Speech (MMS): English Text-to-Speech

This repository contains the English (eng) language text-to-speech (TTS) model checkpoint.

This model is part of Facebook's Massively Multilingual Speech project, aiming to provide speech technology across a diverse range of languages. You can find more details about the supported languages and their ISO 639-3 codes in the MMS Language Coverage Overview, and see all MMS-TTS checkpoints on the Hugging Face Hub: facebook/mms-tts.

MMS-TTS is available in the 🤗 Transformers library from version 4.33 onwards.

Model Details

VITS (Variational Inference with adversarial learning for end-to-end Text-to-Speech) is an end-to-end speech synthesis model that predicts a speech waveform conditional on an input text sequence. It is a conditional variational autoencoder (VAE) comprised of a posterior encoder, decoder, and conditional prior.

A set of spectrogram-based acoustic features are predicted by the flow-based module, which is formed of a Transformer-based text encoder and multiple coupling layers. The spectrogram is decoded using a stack of transposed convolutional layers, much in the same style as the HiFi-GAN vocoder. Motivated by the one-to-many nature of the TTS problem, where the same text input can be spoken in multiple ways, the model also includes a stochastic duration predictor, which allows the model to synthesise speech with different rhythms from the same input text.

The model is trained end-to-end with a combination of losses derived from variational lower bound and adversarial training. To improve the expressiveness of the model, normalizing flows are applied to the conditional prior distribution. During infer

Your obligations as a mms-tts-eng deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Library