KaedeTai commited on
Commit
97877d3
·
verified ·
1 Parent(s): 78c877d

fix: 16.1 GB resident (measured), not 15.5 GB of disk weights

Browse files
Files changed (1) hide show
  1. README.md +3 -1
README.md CHANGED
@@ -52,7 +52,9 @@ Stacked against no speculation at all, on the same target and machine:
52
  | + bf16 drafter | 61.4 | 1.84× |
53
  | **+ Q4 drafter** | **76.1** | **2.29×** |
54
 
55
- A 27B **dense** model at 76 tok/s in ~15.5 GB.
 
 
56
 
57
  ## Target-model comparison (same protocol)
58
 
 
52
  | + bf16 drafter | 61.4 | 1.84× |
53
  | **+ Q4 drafter** | **76.1** | **2.29×** |
54
 
55
+ A 27B **dense** model at 76 tok/s in **16.1 GB resident** (measured RSS with the server
56
+ loaded and generating; 14 GB target + 1.2 GB drafter on disk, plus KV cache — it grows
57
+ with context).
58
 
59
  ## Target-model comparison (same protocol)
60