Speaker wavLM tbr EU AI Act Compliance Profile

Timbral Embeddings extractor

This model produces embeddings that globally represent the timbral traits of a speaker's voice. These embeddings can be used the same way as for a classical speaker verification (ASV): in order to compare two voice signals, an embeddings vector must be computed for each of them. Then the cosine similarity between the two embeddings can be used for comparison. The main difference with classical ASV embeddings is that, here, only the timbral traits are compared.

The model has been derived from the self-supervised pretrained model WavLM-large.

The next section explains how to compute these timbral embeddings.

Usage

The following code snippet uses the file spk_embeddings.py to build the architecture of the model. Its weights are then downloaded from this repository.

The model produces normalized vectors as embeddings.

The python file also contains the function to compute the timbral embeddings of an audio file. In this tutorial version, the audio file is expected to be sampled at 16kHz. Depending on the available memory (cpu or gpu), you may change the value of the max_size parameter, which is used to truncate the long audio signals.

finally, we can compute two embeddings from two different files and compare them with a cosine similarity:

Evaluations

Although it is not directly designed for this use case, evaluation on a standard ASV task can be performed with

Your obligations as a Speaker wavLM tbr deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Languages

Base Model