soda 4b base EU AI Act Compliance Profile

🥤 SODA-4B-base

SODA (Scaling Open Discrete Audio) is a suite of discrete audio foundation models trained from scratch using next-token prediction on interleaved semantic, acoustic, and text tokens.

Built on Marin, SODA started with a simple question: "If we tokenize large-scale audio data into discrete tokens, can we train a standard autoregressive transformer—in the same way we train text LLMs—to build a unified audio backbone where every audio-text task is just next-token prediction?" SODA is a unified pre-trained audio backbone where an audio-language task can be formulated as next-token prediction—through prompting and/or fine-tuning. More details in our project page and/or paper.

This is the 4B model trained on 500B tokens.

• 🌐 Project Page: https://soda-audio.github.io
• 📄 Paper: Scaling Open Discrete Audio Foundation Models with Interleaved Semantic, Acoustic, and Text Tokens
• 💻 Training Code: GitHub
• 🗂️ All Models & Data: soda-research on HuggingFace

Model Details

| | | |---|---| | Parameters | 4B (non-embedding)* | | Training Tokens | 500B | | Training Data | 95% Speech (Yodas + Emilia) + 5% Text (Nemotron-CC) | | Tokenizer | soda-research/marin-mimi-bpe-8cb-16k-tokenizer | | Architecture | Qwen3-based Transformer (cold-start, random initialization) |

*Excludes embedding layers for our scaling law analysis, following the methodology in Kaplan et al. (2020).

Note: SODA was trained exclusively on English speech data and does not currently support other languages.

Unified Next-Token Prediction

SODA treats all audio/text tasks as next-token prediction under a single archit

Your obligations as a soda 4b base deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Languages