Qwen3 0.6B diffusion mdlm v0.1 EU AI Act Compliance Profile

Qwen3-0.6B-diffusion-mdlm-v0.1

Qwen3-0.6B-diffusion-mdlm-v0.1 is a diffusion-based language model adapted from Qwen3-0.6B using MDLM (masked diffusion), trained with the dLLM framework.

Model Overview

Qwen3-0.6B-diffusion-mdlm-v0.1 has the following features:

• Method: Masked Diffusion Language Modeling (MDLM)
• Framework: dLLM
• Base Model: Qwen3-0.6B
• Datasets: tulu-3-sft-mixture, smoltalk, opc-sft-stage1 and opc-sft-stage2

For training details, see the W&B report.

Installation

pip install torch transformers accelerate

Quick Start

[!NOTE] We recommend setting enable_thinking=False when using the model to ensure stable behavior and reproducible results.

import torch
import numpy as np
import torch.nn.functional as F

from transformers import AutoTokenizer, AutoModelForMaskedLM


def add_gumbel_noise(logits, temperature):
    if temperature == 0:
        return logits
    logits = logits.to(torch.float64)
    noise = torch.rand_like(logits, dtype=torch.float64)
    gumbel_noise = (- torch.log(noise)) ** temperature
    return logits.exp() / gumbel_noise


def get_num_transfer_tokens(mask_index, steps):
    mask_num = mask_index.sum(dim=1, keepdim=True)
    base = mask_num // steps
    remainder = mask_num

Your obligations as a Qwen3 0.6B diffusion mdlm v0.1 deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags