Qwen3-Next-80B-A3B-Instruct-FP8 EU AI Act Compliance Profile

Qwen3-Next-80B-A3B-Instruct-FP8

Over the past few months, we have observed increasingly clear trends toward scaling both total parameters and context lengths in the pursuit of more powerful and agentic artificial intelligence (AI). We are excited to share our latest advancements in addressing these demands, centered on improving scaling efficiency through innovative model architecture. We call this next-generation foundation models Qwen3-Next.

[!Note] This repository contains the FP8-quantized Qwen3-Next-80B-A3B-Instruct model checkpoint for convenience and performance. The quantization method is "fine-grained fp8" quantization with block size of 128. You can find more details in the quantization_config field in config.json.

In addition, the experimental results presented in this model card are obtained from the original bfloat16 model prior to FP8 quantization.

Highlights

Qwen3-Next-80B-A3B-FP8 is the first installment in the Qwen3-Next series and features the following key enchancements:

• Hybrid Attention: Replaces standard attention with the combination of Gated DeltaNet and Gated Attention, enabling efficient context modeling for ultra-long context length.
• High-Sparsity Mixture-of-Experts (MoE): Achieves an extreme low activation ratio in MoE layers, drastically reducing FLOPs per token while preserving model capacity.
• Stability Optimizations: Includes techniques such as zero-centered and weight-decayed layernorm, and other stabilizing enhancements for robust pre-training and post-training.
• Multi-Token Prediction (MTP): Boosts pretraining model performance and accelerates inference.

We are seeing strong performance in terms of both parameter efficiency an

Your obligations as a Qwen3-Next-80B-A3B-Instruct-FP8 deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Library

Base Model