ModelsLab/Qwen-Image-2.1-W4A4-nvfp4

Qwen-Image-2.1 W4A4 (nvfp4)

Built with Qwen.

Non-commercial use only. An educational artifact under the Qwen Research License, which the base model carries. Commercial use needs a separate licence from the rights holder; publishing this here does not grant one.

4-bit weights, 4-bit activations, for the Qwen-Image-2.1 transformer. Calibrated with SVDQuant, error-compensated with GPTQ, packed for Nunchaku's fused kernel. 4.54 GB against about 10 GB for the same layers in bf16.

The low-rank branch is fused into the 4-bit GEMM, which is where the speed comes from — an unfused branch gives back 24–34% of it.

Hardware

Blackwell only (sm_120 and up). This will not load on Ada — use the int4 build.

Measured

card	cap	seconds	vs bf16	resident GB	LPIPS vs bf16
RTX PRO 6000 Max-Q	sm_120	7.747	1.71x	21.53	0.1425
RTX 5090	sm_120	7.653	—	21.53	—

40 steps, 1024×1024, seed 42, six prompts. LPIPS is AlexNet against the bf16 image of the same prompt on the same card.

GPTQ is worth 23% here: 0.1858 with round-to-nearest against 0.1425 with GPTQ, at the same speed, the same file size and the same resident memory — it only changes which grid point each weight lands on.

Read LPIPS as distance from the bf16 trajectory, not as damage. On a dense-text prompt every build renders the text correctly while scoring 0.24–0.35, and on one prompt the bf16 reference garbles a sign that this checkpoint renders correctly — and is scored worse for it.

Only the transformer is quantized. The Qwen3-VL text encoder stays bf16 and is about 16.3 GB of a 21.5 GB resident total, so it, not this file, decides how many reference images fit on a card.

Text accuracy against bf16

Every LPIPS number above is measured against bf16, so it cannot answer "is this as good as bf16" — it defines bf16 as zero. This is the reference-free version: 8 seeds on each of two text prompts, scored by reading the rendered text against what the prompt asked for.

build	enamel notice board	neon shop sign	total
bf16	8/8	6/8	14/16
this checkpoint	8/8	7/8	15/16

Indistinguishable. One render's difference out of sixteen, which a different seed set would flip. The errors trade places rather than accumulating on one side: on one seed bf16 renders the sign as "QWEN 2.1 IMAGE 1" and this checkpoint gets it right; on another this checkpoint appends a spurious "1$" where bf16 is clean.

That is the claim worth making — 4-bit weights and activations with no measurable text degradation — and not the stronger one. A single prompt where the quantized build beats bf16 is trajectory variance, not evidence of superiority.

An automated OCR pass was tried first and discarded: tesseract returns noise on neon at night, and a character-error-rate above 1.0 was the giveaway that it was scoring incidental scene text rather than the sign. The table above comes from reading all 48 renders.

samples/ holds one render per prompt at seed 42. seeds/ holds the full sweep.

Requirements

Packed for a specific kernel. The layout is an MMA fragment swizzle, so a different Nunchaku build may read it incorrectly — and would produce a wrong image rather than an error. The loader checks and warns.

torch==2.12.1
nunchaku==1.3.0.dev20260306+cu13.0torch2.12

Loading

import torch
from diffusers import QwenImage21Pipeline
from quantize import nunchaku_io

transformer = nunchaku_io.load_checkpoint(precision="nvfp4", device="cuda")
pipe = QwenImage21Pipeline.from_pretrained(
    "Qwen/Qwen-Image-2.1", torch_dtype=torch.bfloat16, transformer=transformer)

nunchaku_io and the build pipeline: https://github.com/ModelsLab/qwen-image-2-1-server

How it was built

Smoothing factors from real activation statistics (SmoothQuant, α=0.5), a rank-128 SVD branch kept in bf16, and the residual rounded onto Nunchaku's grid by GPTQ against a calibration Hessian. One Hessian pass, 128 distinct inputs across 224 layers, 24 GB resident, ~5 minutes.

w4a4_build.json records the whole build, including per-layer GPTQ statistics.

License

Derivative of Qwen/Qwen-Image-2.1 under the Qwen Research License Agreement — non-commercial only. See NOTICE.