bupalinyu commited on
Commit
d5e59cd
·
verified ·
1 Parent(s): 1480e54

docs: compare the expert pool at FP4 on both sides

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -413,7 +413,7 @@ and how much memory a machine needs:
413
  |---|---:|---:|---:|
414
  | Routed expert parameters | 120.8 B | **84.6 B** (−30%) | **60.4 B** (−50%) |
415
  | Expert parameters per token | 2.36 B | **1.65 B** | **1.18 B** |
416
- | Expert pool on disk · Q3_K builds | 46.84 GiB | **37.32 GiB** | **26.66 GiB** |
417
  | Decode · 16 GiB VRAM + 20 GiB RAM | 18.5 tok/s | **36.8 tok/s** | **62.4 tok/s** |
418
 
419
  On that budget, decode runs 2.0× Qwen's on the 145B and 3.4× on the 120B. Quality holds through the
 
413
  |---|---:|---:|---:|
414
  | Routed expert parameters | 120.8 B | **84.6 B** (−30%) | **60.4 B** (−50%) |
415
  | Expert parameters per token | 2.36 B | **1.65 B** | **1.18 B** |
416
+ | Expert pool on disk · NVFP4 builds | 63.2 GiB | **44.3 GiB** | **31.6 GiB** |
417
  | Decode · 16 GiB VRAM + 20 GiB RAM | 18.5 tok/s | **36.8 tok/s** | **62.4 tok/s** |
418
 
419
  On that budget, decode runs 2.0× Qwen's on the 145B and 3.4× on the 120B. Quality holds through the