aryan5v commited on
Commit
a652472
·
verified ·
1 Parent(s): b03d870

[docs]: make the V2.1 model card self-contained

Browse files

Describe V2.1 and its QAD recipe directly; remove the V2 comparison table and the earlier-run narrative.

Files changed (1) hide show
  1. README.md +19 -23
README.md CHANGED
@@ -5,36 +5,32 @@ tags: [video-generation, text-to-video, audio, fastvideo, nvfp4, qad]
5
  ---
6
  # FastH3 V2.1, 8-step, NVFP4 for one GPU (QAD)
7
 
8
- FastH3 V2.1 is FastH3 V2 (8 distilled steps, video with synchronized audio) with NVFP4 weights recovered by
9
- **quantization-aware distillation (QAD)**. It is a drop-in replacement for
10
- [FastVideo/FastVideo-FastH3-8-Step-V2-NVFP4-Consumer](https://huggingface.co/FastVideo/FastVideo-FastH3-8-Step-V2-NVFP4-Consumer):
11
- same layout, same speed, same single Blackwell GPU (RTX 5090, RTX PRO 6000) or DGX Spark target. Only
12
- `transformer/nvfp4_weights.safetensors` and this card differ.
13
-
14
- - **Transformer:** FastH3 V2 (50 blocks), NVFP4 W4A4 on all attention q/k/v/out and MLP linears, plus the sparse-attention
15
- (VSA) gates. The q/k/v/out and MLP weights were trained with QAD. The VSA gates and all norms are V2's, byte-identical to the
16
- Consumer release. Attention itself runs in bf16 sparse attention (VSA, tile 64, sparsity 0.8), as in V2.
17
  - **Text encoder:** NVFP4 Qwen3-VL trimmed to the 50 layers H3 reads.
18
  - **VAE:** LynnReal lightweight video VAE with Kijai's INT8 weights; H3 audio VAE.
19
  - **Sampling:** `fastvideo_inference.json` holds the 8-step DMD schedule, which FastVideo reads automatically.
20
 
21
- ## What changed from V2 NVFP4
22
-
23
- QAD trains the NVFP4 student to match the bf16 V2 teacher, step by step along the teacher's 8-step trajectory (data-free:
24
- text prompts only, 300 updates, 8 prompts per update, lr 5e-6, 25% of rows on the student's own trajectory). Only the
25
- quantized linears were trained. The VSA gates and norms were frozen: training the gates improved the metric but caused
26
- ghosting on fast motion.
27
 
28
- Per-forward error against the bf16 teacher on 30 held-out prompts (relative L2 of the x0 prediction, lower is better):
 
 
 
 
29
 
30
- | | V2 NVFP4 | **V2.1 (QAD)** |
31
- |---|---|---|
32
- | Mean (video + audio) | 0.431 | **0.351** |
33
- | Video | 0.471 | 0.382 |
34
- | Audio | 0.391 | 0.319 |
35
 
36
- Side-by-side renders on a DGX Spark (11 prompts: faces, fast motion, dance, crowds) show no new artifacts versus V2 NVFP4, with
37
- equal or slightly better faces and fine detail. End-to-end time is unchanged.
 
 
 
38
 
39
  ## Machines with less than 64 GB of system RAM
40
 
 
5
  ---
6
  # FastH3 V2.1, 8-step, NVFP4 for one GPU (QAD)
7
 
8
+ FastH3 V2.1 is an 8-step video-and-audio model with NVFP4 weights, trained with **quantization-aware distillation
9
+ (QAD)**. It runs on one Blackwell GPU (RTX 5090, RTX PRO 6000) or a DGX Spark.
10
+
11
+ - **Transformer:** 50 blocks, NVFP4 W4A4 on all attention q/k/v/out and MLP linears, plus the sparse-attention
12
+ (VSA) gates. The q/k/v/out and MLP weights were trained with QAD. The VSA gates and all norms stay in BF16.
13
+ - **Attention:** block-sparse VSA with 64-token tiles and sparsity 0.8. The checkpoint was trained with BF16 VSA.
14
+ FastVideo also exposes an FP4 VSA inference lane on Blackwell.
 
 
15
  - **Text encoder:** NVFP4 Qwen3-VL trimmed to the 50 layers H3 reads.
16
  - **VAE:** LynnReal lightweight video VAE with Kijai's INT8 weights; H3 audio VAE.
17
  - **Sampling:** `fastvideo_inference.json` holds the 8-step DMD schedule, which FastVideo reads automatically.
18
 
19
+ ## Quantization-aware distillation
 
 
 
 
 
20
 
21
+ QAD trains the NVFP4 student against a frozen BF16 teacher along the teacher's 8-step trajectory. The student runs
22
+ the deployed 4-bit weights and activations, so it learns weights that work with the quantized forward pass. Training
23
+ uses text prompts only: 300 steps, learning rate 5e-6, with 25% of rows on the student's own trajectory. Only the
24
+ quantized linears train. The VSA gates and norms are frozen because changing them changes which video tiles the
25
+ model attends to, not just the arithmetic of attention.
26
 
27
+ Error against the BF16 teacher on 30 held-out prompts (relative L2 of the x0 prediction, lower is better):
 
 
 
 
28
 
29
+ | Metric | V2.1 (QAD) |
30
+ |---|---:|
31
+ | Mean (video + audio) | 0.351 |
32
+ | Video | 0.382 |
33
+ | Audio | 0.319 |
34
 
35
  ## Machines with less than 64 GB of system RAM
36