GLM-4.7-Flash-NVFP4 EU AI Act Compliance Profile

Note: If you have a multi-GPU SM120 Blackwell system (RTX 50/Pro), try my vLLM fork to resolve P2P / TP=2 issues (Pending PR into upstream).

https://github.com/Gadflyii/vllm/tree/main

GLM-4.7-Flash NVFP4 (Mixed Precision)

This is a mixed precision NVFP4 quantization of zai-org/GLM-4.7-Flash, a 30B-A3B (30B total, 3B active) Mixture-of-Experts model.

Quantization Strategy

This model was made via custom quantization and calibration (128 samples, 2048 max seq len, neuralmagic/calibration, all 64 experts) scripts based on NVIDIA's approach for DeepSeek-V3. It uses mixed precision to preserve accuracy:

| Component | Precision | Rationale | |-----------|-----------|-----------| | MLP Experts | FP4 (E2M1) | 64 routed experts, 4 active per token | | Dense MLP | FP4 (E2M1) | First layer dense MLP | | Attention (MLA) | BF16 | Low-rank compressed Q/KV projections are sensitive | | Norms, Gates, Embeddings | BF16 | Standard practice |

Performance

| Metric | BF16 | Uniform FP4 | This Model | |--------|------|-------------|----------------| | MMLU-Pro | 24.83% | 16.84% | 23.55% | | Size | 62.4 GB | 18.9 GB | 20.4 GB | | Compression | 1x | 3.3x | 3.1x | | Accuracy Loss | - | -8.0% | -1.3% |

Usage

Requirements

• vLLM: 0.14.0+ (for compressed-tensors NVFP4 support)
• transformers: 5.0.0+ (for glm4_moe_lite architecture)
• GPU: NVIDIA GPU with FP4 tensor core support (Blackwell, Hopper, Ada Lovelace)

Installation

pip install vllm>=0.14.0
pip install git+https://github.com/huggingface/transformers.git

Inference with vLLM

from vllm import LLM, SamplingParams

model = LLM(
    "GadflyII/GLM-4.7-Flash-NVFP4",
    tensor_parallel_size=1,
    max_model_len=4096,
    trust_remote_code=True,
    gpu_memory_utilization=0.85,
)

params = SamplingParams(temperature=0.7, max_tokens=512)
outputs = model.generate(["Explain quantum computin

Your obligations as a GLM-4.7-Flash-NVFP4 deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Languages

Library

Base Model