Fast dLLM v2 1.5B EU AI Act Compliance Profile

Fast-dLLM v2 (1.5B) — Efficient Block-Diffusion LLM

📖 Introduction

Autoregressive (AR) large language models (LLMs) have achieved remarkable performance across a wide range of natural language tasks, yet their inherent sequential decoding limits inference efficiency.

We present Fast-dLLM v2 — a carefully designed block diffusion language model (dLLM) that efficiently adapts a pretrained AR model (Qwen2.5-1.5B-Instruct) into a diffusion-style decoder for parallel text generation.

Our approach introduces a novel decoding recipe incorporating a complementary attention mask and block diffusion mechanism, which together enable blockwise bidirectional context modeling while preserving the original AR training objectives and performance. To further enhance inference speed, we design a hierarchical caching mechanism: a block-level cache that stores historical context representations and a sub-block level cache that supports efficient parallel decoding within partially generated blocks.

✨ Key Innovations

• Block Diffusion Mechanism + Complementary Attention Mask
  Enables blockwise bidirectional context modeling without sacrificing AR objectives.
• Hierarchical Caching
  • Block-level cache: Stores historical context representations across blocks.
  • Sub-block cache: Parallel decoding within partially generated blocks.
• Token Shift Mechanism
  Retains autoregressive characteristics while supporting bidirectional context within blocks.
• Parallel Decoding Pipeline
  Achieves up to 2.5× speedup over standard AR decoding without compromising quality.

🚀 Fast-dLLM v2 uses only ~1B tokens for fine-tuning — a 500× reduction vs. full-attention diffusion LLMs (Dream: 580B tokens) — while matching or surpassing AR baselines in accuracy.

🛠 Model Overview

• Type: Block Diffusion Language Model (dLLM)
• Base Model:

Your obligations as a Fast dLLM v2 1.5B deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Languages

Base Model