Instructions to use intel-ai/Wan2.2-T2V-A14B-Diffusers-NVFP4-E5M3-AutoRound with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use intel-ai/Wan2.2-T2V-A14B-Diffusers-NVFP4-E5M3-AutoRound with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("intel-ai/Wan2.2-T2V-A14B-Diffusers-NVFP4-E5M3-AutoRound", dtype=torch.bfloat16, device_map="cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - Wan2.2
How to use intel-ai/Wan2.2-T2V-A14B-Diffusers-NVFP4-E5M3-AutoRound with Wan2.2:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Wan2.2-T2V-A14B-Diffusers-NVFP4-E5M3-AutoRound
This checkpoint is an AutoRound NVFP4_E5M3 quantized version of Wan-AI/Wan2.2-T2V-A14B-Diffusers, a text-to-video model with high-noise and low-noise diffusion experts.
Both diffusion transformers are quantized using model-free round-to-nearest (RTN), with no calibration dataset, no calibration forward passes, and no tuning iterations. Output projections remain unquantized to align the layer scope with the ordinary NVFP4 comparison.
Evaluation on a fixed 165-prompt, 720P VBench subset shows absolute decreases of 0.6133 percentage points in imaging quality and 0.8903 percentage points in aesthetic quality versus BF16. Motion smoothness increases slightly; overall and subject consistency decrease by less than 0.3 percentage points each. These are individual dimension scores, not an official VBench total.
Runtime requirement: NVFP4_E5M3 uses FP4 Marlin weight GEMM with E5M3 activation QDQ, not a native ordinary-NVFP4 W4A4 GEMM. Install the pinned plugin below; no vLLM-Omni source changes or external adapter are needed.
Model Details
| Item | Value |
|---|---|
| Base model | Wan-AI/Wan2.2-T2V-A14B-Diffusers |
| Task | Text-to-video generation |
| Quantization tool | AutoRound, model-free RTN |
| Scheme | NVFP4_E5M3 |
| Weight / activation format | FP4 E2M1 values with unsigned E5M3 block scales |
| Weight / activation group size | 16 / 16 |
| Activation quantization | Dynamic, 4-bit; QDQ in the validated runtime |
| Quantized components | transformer and transformer_2 |
| Quantized scope | 400 block linear modules per transformer; 800 total |
| Output projection | proj_out excluded from quantization in both transformers |
| Calibration data / tuning iterations | None / 0 |
| Export format | AutoRound; auto_round:llm_compressor_nvfp4_e5m3 |
| Serialized datatype identifier | nvfp4_v2 |
| Validated pipeline | WanPipeline through the vLLM-Omni Python API |
| Validated resolution / hardware | 1280 × 720 / NVIDIA B200 |
| License | Apache-2.0 |
The text encoder and VAE are not quantized by this recipe. The results below apply to the stated 720P settings, not to all resolutions or generation configurations.
Output-projection preservation
The initial model-free export also quantized proj_out. For the checkpoint evaluated here, both experts' proj_out.weight and proj_out.bias were restored byte-for-byte from the original base checkpoint, not reconstructed by dequantizing FP4 weights. All other exported tensors were hash-verified as unchanged.
The restored output projections retain the source checkpoint's FP32 on-disk storage and are cast through the BF16 inference path at runtime. The quantization configuration excludes them using bits=16 and act_bits=16; these settings do not imply BF16 storage on disk. Block quantization remains unchanged.
How to Use
Environment
Use the recorded NVIDIA B200 environment and official vLLM-Omni commit 7465c0f52. Install the plugin from the exact tested commit:
python -m pip install --no-deps "git+https://github.com/intel/neural-compressor.git@8221432d6c328fdb39264bf059a02700bb753059#subdirectory=benchmark/vllm-qdq-plugin"
The inc_nvfp4_e5m3 entry point is discovered automatically. If using a plugin allowlist, include this name alongside any other required plugins. VLLM_QDQ=0 disables unrelated simulation, not E5M3's required activation QDQ. No manual adapter registration is needed.
Text-to-Video Example
From the pinned vLLM-Omni checkout, run the official example with the same boxing-cat prompt and parameters used in the BF16/MXFP8/NVFP4 comparison cards. Set MODEL to the downloaded checkpoint directory.
MODEL=./wan22_t2v_nvfp4_e5m3_projout_bf16_v2
CUDA_VISIBLE_DEVICES=0 DIFFUSION_ATTENTION_BACKEND=TRTLLM_ATTN \
VLLM_QDQ=0 VLLM_PLUGINS=inc_nvfp4_e5m3 \
python examples/offline_inference/text_to_video/text_to_video.py \
--model "$MODEL" --model-class-name WanPipeline \
--prompt "Two anthropomorphic cats in comfy boxing gear and bright gloves fight intensely on a spotlighted stage." \
--negative-prompt "色调艳丽,过曝,静态,细节模糊不清,字幕,风格,作品,画作,画面,静止,整体发灰,最差质量,低质量,JPEG压缩残留,丑陋的,残缺的,多余的手指,画得不好的手部,画得不好的脸部,畸形的,毁容的,形态畸形的肢体,手指融合,静止不动的画面,杂乱的背景,三条腿,背景人很多,倒着走" \
--height 720 --width 1280 --num-frames 81 --fps 16 \
--num-inference-steps 40 --guidance-scale 4.0 --guidance-scale-high 3.0 \
--boundary-ratio 0.875 --flow-shift 5.0 --seed 42 --enforce-eager \
--output nvfp4_e5m3.mp4
Video Generation Comparison
The E5M3 boxing-cat showcase was generated with plugin commit 8221432d6c328fdb39264bf059a02700bb753059 using the command above. It uses the same prompt, negative prompt, resolution, seed and sampling settings as the previous cards: 1280 × 720, 81 frames, 16 fps. Generation exited successfully and all frames passed decoding validation. This individual showcase is separate from the benchmark.
| BF16 Reference | NVFP4_E5M3 (This Model) |
|---|---|
| Download BF16 video | Download NVFP4_E5M3 video |
Validation provenance
Both tasks passed one-video end-to-end smoke tests with plugin commit 8221432d6c328fdb39264bf059a02700bb753059, automatic entry-point loading and unchanged Omni source. The full VBench scores below were obtained earlier with plugin 9d95229f7e18c5e403b470934447134a7d611452 plus an external Omni adapter. The full benchmark has not been rerun with the new plugin.
Recorded validation environment
| Component | Recorded version |
|---|---|
| Python | 3.12.3 |
| PyTorch | 2.14.0+cu130 |
| vLLM | 0.33.0 |
| Diffusers | 0.40.0 |
| FlashInfer | 0.7.2.post1 |
| CUTLASS DSL | 4.8.0 |
| VBench | 0.1.5 |
The PyTorch build uses CUDA 13.0; the selected JIT/compiler toolkit was CUDA 12.8.93. These are records of the tested environment, not a clean-install lockfile. The existing overlay also contained Transformers 5.20.2, exceeding the pinned Omni source's <5.20.0 requirement. A fresh dependency-resolved installation has not been validated. Results are not claims of support for other GPU architectures or arbitrary package versions.
Evaluation
BF16 comparison
Scores are multiplied by 100 for readability. Higher is better. Differences are NVFP4_E5M3 minus BF16, expressed in absolute percentage points, not relative percentage loss.
| VBench dimension | Videos | BF16 | NVFP4_E5M3 | Difference (pp) |
|---|---|---|---|---|
| Imaging quality | 165 | 72.1889 | 71.5756 | -0.6133 |
| Aesthetic quality | 165 | 62.2004 | 61.3100 | -0.8903 |
| Motion smoothness | 165 | 98.0194 | 98.1045 | +0.0852 |
| Overall consistency | 165 | 23.7381 | 23.6554 | -0.0827 |
| Subject consistency | 165 | 95.1550 | 94.8917 | -0.2634 |
The larger measured differences are in aesthetic and imaging quality. Motion smoothness is slightly higher in this fixed-seed evaluation; this does not establish a general quality improvement or statistical significance.
Generation protocol
BF16 and NVFP4_E5M3 use the same selected prompts and generation settings:
| Setting | Value |
|---|---|
| Inputs | 165 ordered VBench prompts: 72 subject-consistency prompts and 93 overall-consistency prompts |
| Videos per metric | All 165 videos for each of the five dimensions |
| Output resolution | 1280 × 720 |
| Frames / FPS | 81 / 16 |
| Denoising steps | 40 |
| Guidance scale, high / low noise | 4.0 / 3.0 |
| Seed | 42, reset for each prompt |
| Boundary ratio / flow shift | 0.875 / 5.0 |
| Attention backend | TRTLLM_ATTN |
| Execution | Eager; no diffusion cache, CPU/layerwise offload, or VAE tiling/slicing |
The recorded configuration retains linear_backend=flashinfer_cutlass, but quantized E5M3 INC layers actually use the plugin's Marlin + activation-QDQ path. This configuration value is not evidence of native FlashInfer NVFP4 execution.
Generation used independent deterministic shards on two GPUs, not tensor parallelism. All 165/165 videos passed integrity and format verification before scoring. All five metrics completed successfully.
Exact negative prompt used for both precisions
色调艳丽,过曝,静态,细节模糊不清,字幕,风格,作品,画作,画面,静止,整体发灰,最差质量,低质量,JPEG压缩残留,丑陋的,残缺的,多余的手指,画得不好的手部,画得不好的脸部,畸形的,毁容的,形态畸形的肢体,手指融合,静止不动的画面,杂乱的背景,三条腿,背景人很多,倒着走
Scoring protocol and scope
- VBench 0.1.5, using its official implementations of imaging quality, aesthetic quality, motion smoothness, overall consistency, and subject consistency.
local=True,read_frame=False, and imaging-quality preprocessing modelonger.- Every dimension is evaluated on all 165 videos, not just the prompts originally tagged for that dimension.
- One generated video per prompt at a fixed seed; videos are not duplicated to mimic multiple samples.
- This is a selected single-sample T2V evaluation, not the standard five-sample full benchmark. No official VBench total is reported.
- Matching generation and scoring settings does not imply matching quantized kernels or bitwise-identical execution.
Exact NVFP4_E5M3 scores on the original 0–1 scale
{
"imaging_quality": 0.7157555346619178,
"aesthetic_quality": 0.6131004111333327,
"motion_smoothness": 0.9810453448442675,
"overall_consistency": 0.2365539898249236,
"subject_consistency": 0.94891676662772
}
Generate the Model
The initial model-free export used AutoRound at commit a85997e19d97762172155c93c4dfc3fcc3976e71 (export metadata version 0.17.0):
python -m auto_round \
--model Wan-AI/Wan2.2-T2V-A14B-Diffusers \
--scheme NVFP4_E5M3 \
--model_free \
--algorithm rtn \
--disable_opt_rtn \
--iters 0 \
--device_map 0 \
--format auto_round \
--output_dir ./wan22_t2v_nvfp4_e5m3_model_free
This command documents the initial export, not a turnkey evaluated checkpoint. It must be followed by byte-exact restoration of both experts' proj_out.weight and proj_out.bias from the original base checkpoint, configuration/index updates, and hash verification that all other exported tensors are unchanged, as described above. Running this command alone does not reproduce the evaluated layer scope. No dataset or calibration input is required for this model-free recipe.
The Wan2.2 quantization/VBench repository documents shared benchmark inputs and the historical comparison protocol. The E5M3 runtime extensions and complete reproduction workflow are not asserted to be available on its published main branch.
Limitations
- Quantization can alter appearance, prompt adherence, temporal consistency, and motion. Inspect outputs for the intended application.
- No inference speedup, minimum VRAM requirement, or hardware-independent memory-reduction figure is claimed; these were not measured in a controlled comparison here.
- The reported evaluation does not establish robustness across seeds, performance on other prompt distributions, or quality at other resolutions and generation lengths.
- Original base-model demos or ordinary-NVFP4 videos are not presented as outputs from this E5M3 checkpoint.
- Follow the base model's usage guidance and applicable laws, respect rights and consent, and disclose synthetic media where appropriate.
References
Signed-off-by: changwangss chang1.wang@intel.com
- Downloads last month
- 41
Model tree for intel-ai/Wan2.2-T2V-A14B-Diffusers-NVFP4-E5M3-AutoRound
Base model
Wan-AI/Wan2.2-T2V-A14B-Diffusers