moondream3 preview EU AI Act Compliance Profile

Moondream 3 (Preview) is an vision language model with a mixture-of-experts architecture (9B total parameters, 2B active). This model makes no compromises, delivering state-of-the-art visual reasoning while still retaining our efficient and deployment-friendly ethos.

✨ Demo   ·   ☁️ Cloud API   ·   📝 Release notes

Architecture

1. 24 layers; the first four are dense, the rest have MoE FFNs with 64 experts, 8 activated per token
2. MoE FFNs have GeGLU architecture, with inner/gate dim of 1024. The model's hidden dim is 2048.
3. Usable context length increased to 32K, with a custom efficient SuperBPE tokenizer
4. Multi-headed attention with learned position- and data-dependent temperature scaling
5. SigLIP-based vision encoder, with multi-crop channel concatenation for token-efficient high resolution image processing

For more details, please refer to the [release notes]((https://moondream.ai/blog/moondream-3-preview). Or try the model out in our playground demo.

The following instructions demonstrate how to run the model locally using Transformers. We also offer a cloud API with a generous free tier that can help you get started quicker!

Usage

Load the model and prepare it for inference. We use FlexAttention for inference, so calling .compile() is critical for fast

Your obligations as a moondream3 preview deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Library