Speaker wavLM pro EU AI Act Compliance Profile

Non-timbral Embeddings extractor

This model produces embeddings that globally represent the non-timbral traits (prosody, accent, ...) of a speaker's voice. These embeddings can be used the same way as for a classical speaker verification (ASV): in order to compare two voice signals, an embeddings vector must be computed for each of them. Then the cosine similarity between the two embeddings can be used for comparison. The main difference with classical ASV embeddings is that here only the non-timbral traits are compared.

The model has been derived from the self-supervised pretrained model WavLM-large.

The next section explains how to compute these non-timbral embeddings.

Usage

The following code snippet uses the file spk_embeddings.py to build the architecture of the model. Its weights are then downloaded from this repository.

The model produces normalized vectors as embeddings.

The python file also contains the function to compute the non-timbral embedding of an audio file. In this tutorial version, the audio file is expected to be sampled at 16kHz. Depending on the available memory (cpu or gpu), you may change the value of the max_size parameter, which is used to truncate the long audio signals.

finally, we can compute two embeddings from two different files and compare them with a cosine similarity:

Evaluations

Although it is not directly designed for this use case, evaluation on a s

Your obligations as a Speaker wavLM pro deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Languages

Base Model