Frame VAD Multilingual MarbleNet v2.0 EU AI Act Compliance Profile

Frame-VAD Multilingual MarbleNet v2.0

Description:

Frame-VAD Multilingual MarbleNet v2.0 is a convolutional neural network for voice activity detection (VAD) that serves as the first step for Speech Recognition and Speaker Diarization. It is a frame-based model that outputs a speech probability for each 20 millisecond frame of the input audio. The model has 91.5K parameters, making it lightweight and efficient for real-time applications. <br> To reduce false positive errors — cases where the model incorrectly detects speech when none is present — the model was trained with white noise and real-word noise perturbations. During training, the volume of audios was also varied. Additionally, the training data includes non-speech audio samples to help the model distinguish between speech and non-speech sounds (such as coughing, laughter, and breathing, etc.) <br>

Key Features

• Lightweight model with only 91.5K parameters
• Robust against false positive errors
• Outputs speech probability for each 20 ms audio frame
• Multilingual support: Chinese, English, French, German, Russian, and Spanish

This model is ready for commercial use. <br>

License/Terms of Use:

GOVERNING TERMS: Your use of this model is governed by the NVIDIA Open Model License Agreement.

Deployment Geography:

Global <br>

Use Case:

Developers, speech processing engineers, and AI researchers will use it as the first step for other speech processing models. <br>

References:

[1] Jia, Fei, Somshubra Majumdar, and Boris Ginsburg. "MarbleNet: Deep 1D Time-Channel Separable Convolutional Neural Network for Voice Activity Detection." ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2021. <br> [2] NVIDIA NeMo Toolkit <br>

Model Architecture:

*Architecture Type:

Your obligations as a Frame VAD Multilingual MarbleNet v2.0 deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Languages

Library