reyden009's picture
Upload folder using huggingface_hub
d4c2896 verified
|
Raw
History Blame Contribute Delete
1.99 kB

Qwen3-8B — target quant (q4/q5/q8) x drafter

Target quant Drafter n tok/s x vs solo alpha tau
q4 Vanilla-1.7B 1474 75.5 1.41x 0.725 173
q4 EAGLE-3 1474 74.4 1.39x 0.440 141
q4 DFlash-F16 1474 33.4 0.63x 0.009 7
q4 DSpark p=0.0 1474 87.6 1.64x 0.616 162
q4 DSpark p=0.2 1474 86.7 1.62x 0.621 162
q4 DSpark p=0.4 1474 85.4 1.60x 0.646 161
q4 DSpark p=0.6 1474 80.7 1.51x 0.714 154
q5 EAGLE-3 1474 68.3 1.46x 0.439 140
q5 DFlash-F16 1474 30.8 0.66x 0.009 7
q5 DSpark p=0.0 1474 80.6 1.72x 0.613 162
q8 EAGLE-3 1474 52.8 1.63x 0.432 140
q8 DSpark p=0.0 1474 62.3 1.93x 0.611 162

Gemma 4 12B — target quant x drafter

Target quant Drafter n tok/s x vs solo alpha tau
q4 DFlash-F16 1455 72.2 2.09x 0.657 156
q4 DFlash-Q4 1455 78.4 2.26x 0.647 155
q4 DFlash-Q8 1455 76.6 2.21x 0.656 156
q4 MTP 1455 82.7 2.39x 0.739 164
q5 DFlash-F16 1449 64.5 2.16x 0.659 157
q5 MTP 1449 75.6 2.53x 0.745 165
q8 DFlash-F16 1452 45.7 2.30x 0.658 156
q8 MTP 1452 54.0 2.71x 0.745 164

Gemma-4 Q4 — draft quantization effect (DFlash drafts, final runs)

Drafter (draft quant) n tok/s x vs solo alpha tau max VRAM (MiB)
DFlash-F16 1455 72.2 2.09x 0.657 156 10766
DFlash-Q4 1455 78.4 2.26x 0.647 155 9730
DFlash-Q8 1455 76.6 2.21x 0.656 156 10043

F16 = 1.47 GB draft, Q4_K_M = 0.44 GB, Q8_0 = 0.79 GB (model-hashes.json).