mimi EU AI Act Compliance Profile

Model Card for Mimi

Mimi codec is a state-of-the-art audio neural codec, developped by Kyutai, that combines semantic and acoustic information into audio tokens running at 12.5Hz and a bitrate of 1.1kbps.

Model Details

Model Description

Mimi is a high-fidelity audio codec leveraging neural networks. It introduces a streaming encoder-decoder architecture with quantized latent space, trained in an end-to-end fashion.

It was trained on speech data, which makes it particularly adapted to train speech language models or text-to-speech systems.

• Developed by: Kyutai
• Model type: Audio codec
• Audio types: Speech
• License: CC-BY

Model Sources

• Repository: repo
• Paper: paper
• Demo: demo

Uses

How to Get Started with the Model

Usage with transformers

Use the following code to get started with the Mimi model using a dummy example from the LibriSpeech dataset (~9MB). First, install the required Python packages:

Then load an audio sample, and run a forward pass of the model:

from datasets import load_dataset, Audio
from transformers import MimiModel, AutoFeatureExtractor

# load a demonstration datasets
librispeech_dummy = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation")

# load the model + feature extractor (for pre-processing the audio)
model = MimiModel.from_pretrained("kyutai/mimi")
feature_extractor = AutoFeatureExtractor.from_pretrained("kyutai/mimi")

# cast the audio data to the correct sampling rate for the model
librispeech_dummy = librispeech_dummy.cast_column("audio",

Your obligations as a mimi deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Library