whisper large v3 turbo et verbatim EU AI Act Compliance Profile

Introduction

This model is OpenAI Whisper large-v3-turbo, finetuned on 1400 hours of audio with manually created verbatim transcriptions from the TalTech Estonian Speech Dataset 1.0 (https://cs.taltech.ee/staff/tanel.alumae/data/est-pub-asr-data/).

Usage

It's a finetuned vesion of Whisper large-v3-turbo and can be therefore used via Hugging Face 🤗 Transformers. To run the model, first install the Transformers library. For this example, we'll also install 🤗 Accelerate to reduce the model loading time:

pip install --upgrade pip
pip install --upgrade transformers accelerate

The model can be used with the pipeline class to transcribe audios of arbitrary length:

import torch
from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor, pipeline
from datasets import load_dataset


device = "cuda:0" if torch.cuda.is_available() else "cpu"
torch_dtype = torch.float16 if torch.cuda.is_available() else torch.float32

model_id = "TalTechNLP/whisper-large-v3-turbo-et-verbatim"

model = AutoModelForSpeechSeq2Seq.from_pretrained(
model_id, torch_dtype=torch_dtype, low_cpu_mem_usage=True, use_safetensors=True
)
model.to(device)

processor = AutoProcessor.from_pretrained(model_id)

pipe = pipeline(
"automatic-speech-recognition",
model=model,
tokenizer=processor.tokenizer,
feature_extractor=processor.feature_extractor,
torch_dtype=torch_dtype,
device=device,
)

audio = "demo/etteütlus2024.wav"

result = pipe(sample, generate_kwargs={"task": "transcribe", "language": "et"})
print(result)

There is a also a ct2 version of the model that can be used with tools that are based on faster-whisper, e.g. whisper-ctranslate2 command line program, e.g.:

Your obligations as a whisper large v3 turbo et verbatim deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Languages

Library

Base Model