Wan2.2-T2V-A14B-Diffusers-NVFP4-E5M3-AutoRound

This checkpoint is an AutoRound NVFP4_E5M3 quantized version of Wan-AI/Wan2.2-T2V-A14B-Diffusers, a text-to-video model with high-noise and low-noise diffusion experts.

Both diffusion transformers are quantized using model-free round-to-nearest (RTN), with no calibration dataset, no calibration forward passes, and no tuning iterations. Output projections remain unquantized to align the layer scope with the ordinary NVFP4 comparison.

Evaluation on a fixed 165-prompt, 720P VBench subset shows absolute decreases of 0.6133 percentage points in imaging quality and 0.8903 percentage points in aesthetic quality versus BF16. Motion smoothness increases slightly; overall and subject consistency decrease by less than 0.3 percentage points each. These are individual dimension scores, not an official VBench total.

Runtime requirement: NVFP4_E5M3 uses FP4 Marlin weight GEMM with E5M3 activation QDQ, not a native ordinary-NVFP4 W4A4 GEMM. Install the pinned plugin below; no vLLM-Omni source changes or external adapter are needed.

Model Details

Item Value
Base model Wan-AI/Wan2.2-T2V-A14B-Diffusers
Task Text-to-video generation
Quantization tool AutoRound, model-free RTN
Scheme NVFP4_E5M3
Weight / activation format FP4 E2M1 values with unsigned E5M3 block scales
Weight / activation group size 16 / 16
Activation quantization Dynamic, 4-bit; QDQ in the validated runtime
Quantized components transformer and transformer_2
Quantized scope 400 block linear modules per transformer; 800 total
Output projection proj_out excluded from quantization in both transformers
Calibration data / tuning iterations None / 0
Export format AutoRound; auto_round:llm_compressor_nvfp4_e5m3
Serialized datatype identifier nvfp4_v2
Validated pipeline WanPipeline through the vLLM-Omni Python API
Validated resolution / hardware 1280 × 720 / NVIDIA B200
License Apache-2.0

The text encoder and VAE are not quantized by this recipe. The results below apply to the stated 720P settings, not to all resolutions or generation configurations.

Output-projection preservation

The initial model-free export also quantized proj_out. For the checkpoint evaluated here, both experts' proj_out.weight and proj_out.bias were restored byte-for-byte from the original base checkpoint, not reconstructed by dequantizing FP4 weights. All other exported tensors were hash-verified as unchanged.

The restored output projections retain the source checkpoint's FP32 on-disk storage and are cast through the BF16 inference path at runtime. The quantization configuration excludes them using bits=16 and act_bits=16; these settings do not imply BF16 storage on disk. Block quantization remains unchanged.

How to Use

Environment

Use the recorded NVIDIA B200 environment and official vLLM-Omni commit 7465c0f52. Install the plugin from the exact tested commit:

python -m pip install --no-deps "git+https://github.com/intel/neural-compressor.git@8221432d6c328fdb39264bf059a02700bb753059#subdirectory=benchmark/vllm-qdq-plugin"

The inc_nvfp4_e5m3 entry point is discovered automatically. If using a plugin allowlist, include this name alongside any other required plugins. VLLM_QDQ=0 disables unrelated simulation, not E5M3's required activation QDQ. No manual adapter registration is needed.

Text-to-Video Example

From the pinned vLLM-Omni checkout, run the official example with the same boxing-cat prompt and parameters used in the BF16/MXFP8/NVFP4 comparison cards. Set MODEL to the downloaded checkpoint directory.

MODEL=./wan22_t2v_nvfp4_e5m3_projout_bf16_v2
CUDA_VISIBLE_DEVICES=0 DIFFUSION_ATTENTION_BACKEND=TRTLLM_ATTN \
VLLM_QDQ=0 VLLM_PLUGINS=inc_nvfp4_e5m3 \
python examples/offline_inference/text_to_video/text_to_video.py \
  --model "$MODEL" --model-class-name WanPipeline \
  --prompt "Two anthropomorphic cats in comfy boxing gear and bright gloves fight intensely on a spotlighted stage." \
  --negative-prompt "色调艳丽,过曝,静态,细节模糊不清,字幕,风格,作品,画作,画面,静止,整体发灰,最差质量,低质量,JPEG压缩残留,丑陋的,残缺的,多余的手指,画得不好的手部,画得不好的脸部,畸形的,毁容的,形态畸形的肢体,手指融合,静止不动的画面,杂乱的背景,三条腿,背景人很多,倒着走" \
  --height 720 --width 1280 --num-frames 81 --fps 16 \
  --num-inference-steps 40 --guidance-scale 4.0 --guidance-scale-high 3.0 \
  --boundary-ratio 0.875 --flow-shift 5.0 --seed 42 --enforce-eager \
  --output nvfp4_e5m3.mp4

Video Generation Comparison

The E5M3 boxing-cat showcase was generated with plugin commit 8221432d6c328fdb39264bf059a02700bb753059 using the command above. It uses the same prompt, negative prompt, resolution, seed and sampling settings as the previous cards: 1280 × 720, 81 frames, 16 fps. Generation exited successfully and all frames passed decoding validation. This individual showcase is separate from the benchmark.

BF16 Reference NVFP4_E5M3 (This Model)
Download BF16 video Download NVFP4_E5M3 video

Validation provenance

Both tasks passed one-video end-to-end smoke tests with plugin commit 8221432d6c328fdb39264bf059a02700bb753059, automatic entry-point loading and unchanged Omni source. The full VBench scores below were obtained earlier with plugin 9d95229f7e18c5e403b470934447134a7d611452 plus an external Omni adapter. The full benchmark has not been rerun with the new plugin.

Recorded validation environment

Component Recorded version
Python 3.12.3
PyTorch 2.14.0+cu130
vLLM 0.33.0
Diffusers 0.40.0
FlashInfer 0.7.2.post1
CUTLASS DSL 4.8.0
VBench 0.1.5

The PyTorch build uses CUDA 13.0; the selected JIT/compiler toolkit was CUDA 12.8.93. These are records of the tested environment, not a clean-install lockfile. The existing overlay also contained Transformers 5.20.2, exceeding the pinned Omni source's <5.20.0 requirement. A fresh dependency-resolved installation has not been validated. Results are not claims of support for other GPU architectures or arbitrary package versions.

Evaluation

BF16 comparison

Scores are multiplied by 100 for readability. Higher is better. Differences are NVFP4_E5M3 minus BF16, expressed in absolute percentage points, not relative percentage loss.

VBench dimension Videos BF16 NVFP4_E5M3 Difference (pp)
Imaging quality 165 72.1889 71.5756 -0.6133
Aesthetic quality 165 62.2004 61.3100 -0.8903
Motion smoothness 165 98.0194 98.1045 +0.0852
Overall consistency 165 23.7381 23.6554 -0.0827
Subject consistency 165 95.1550 94.8917 -0.2634

The larger measured differences are in aesthetic and imaging quality. Motion smoothness is slightly higher in this fixed-seed evaluation; this does not establish a general quality improvement or statistical significance.

Generation protocol

BF16 and NVFP4_E5M3 use the same selected prompts and generation settings:

Setting Value
Inputs 165 ordered VBench prompts: 72 subject-consistency prompts and 93 overall-consistency prompts
Videos per metric All 165 videos for each of the five dimensions
Output resolution 1280 × 720
Frames / FPS 81 / 16
Denoising steps 40
Guidance scale, high / low noise 4.0 / 3.0
Seed 42, reset for each prompt
Boundary ratio / flow shift 0.875 / 5.0
Attention backend TRTLLM_ATTN
Execution Eager; no diffusion cache, CPU/layerwise offload, or VAE tiling/slicing

The recorded configuration retains linear_backend=flashinfer_cutlass, but quantized E5M3 INC layers actually use the plugin's Marlin + activation-QDQ path. This configuration value is not evidence of native FlashInfer NVFP4 execution.

Generation used independent deterministic shards on two GPUs, not tensor parallelism. All 165/165 videos passed integrity and format verification before scoring. All five metrics completed successfully.

Exact negative prompt used for both precisions
色调艳丽,过曝,静态,细节模糊不清,字幕,风格,作品,画作,画面,静止,整体发灰,最差质量,低质量,JPEG压缩残留,丑陋的,残缺的,多余的手指,画得不好的手部,画得不好的脸部,畸形的,毁容的,形态畸形的肢体,手指融合,静止不动的画面,杂乱的背景,三条腿,背景人很多,倒着走

Scoring protocol and scope

  • VBench 0.1.5, using its official implementations of imaging quality, aesthetic quality, motion smoothness, overall consistency, and subject consistency.
  • local=True, read_frame=False, and imaging-quality preprocessing mode longer.
  • Every dimension is evaluated on all 165 videos, not just the prompts originally tagged for that dimension.
  • One generated video per prompt at a fixed seed; videos are not duplicated to mimic multiple samples.
  • This is a selected single-sample T2V evaluation, not the standard five-sample full benchmark. No official VBench total is reported.
  • Matching generation and scoring settings does not imply matching quantized kernels or bitwise-identical execution.
Exact NVFP4_E5M3 scores on the original 0–1 scale
{
  "imaging_quality": 0.7157555346619178,
  "aesthetic_quality": 0.6131004111333327,
  "motion_smoothness": 0.9810453448442675,
  "overall_consistency": 0.2365539898249236,
  "subject_consistency": 0.94891676662772
}

Generate the Model

The initial model-free export used AutoRound at commit a85997e19d97762172155c93c4dfc3fcc3976e71 (export metadata version 0.17.0):

python -m auto_round \
  --model Wan-AI/Wan2.2-T2V-A14B-Diffusers \
  --scheme NVFP4_E5M3 \
  --model_free \
  --algorithm rtn \
  --disable_opt_rtn \
  --iters 0 \
  --device_map 0 \
  --format auto_round \
  --output_dir ./wan22_t2v_nvfp4_e5m3_model_free

This command documents the initial export, not a turnkey evaluated checkpoint. It must be followed by byte-exact restoration of both experts' proj_out.weight and proj_out.bias from the original base checkpoint, configuration/index updates, and hash verification that all other exported tensors are unchanged, as described above. Running this command alone does not reproduce the evaluated layer scope. No dataset or calibration input is required for this model-free recipe.

The Wan2.2 quantization/VBench repository documents shared benchmark inputs and the historical comparison protocol. The E5M3 runtime extensions and complete reproduction workflow are not asserted to be available on its published main branch.

Limitations

  • Quantization can alter appearance, prompt adherence, temporal consistency, and motion. Inspect outputs for the intended application.
  • No inference speedup, minimum VRAM requirement, or hardware-independent memory-reduction figure is claimed; these were not measured in a controlled comparison here.
  • The reported evaluation does not establish robustness across seeds, performance on other prompt distributions, or quality at other resolutions and generation lengths.
  • Original base-model demos or ordinary-NVFP4 videos are not presented as outputs from this E5M3 checkpoint.
  • Follow the base model's usage guidance and applicable laws, respect rights and consent, and disclose synthetic media where appropriate.

References

Signed-off-by: changwangss chang1.wang@intel.com

Downloads last month
41
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for intel-ai/Wan2.2-T2V-A14B-Diffusers-NVFP4-E5M3-AutoRound

Quantized
(21)
this model

Collection including intel-ai/Wan2.2-T2V-A14B-Diffusers-NVFP4-E5M3-AutoRound