Hugging Face's logo

Spaces:
ai-engineering-at
/
llama-cpp-turboquant-guide
Running

App Files Community
AI Engineering Lab commited on
Commit
bf29ad1
·
1 Parent(s): 78e199b

results: consolidate 4070 Laptop — 2 independent sessions, avg -4.6% TPS

Browse files
Files changed (1) hide show
  1. README.md +4 -4
README.md CHANGED
@@ -69,17 +69,17 @@ Raw data: [`results/turboquant-rtx3090-2026-04-01.json`](results/turboquant-rtx3
69
 
70
  ### RTX 4070 Laptop (8 GB) — Llama-3.1-8B-Instruct Q4_K_M
71
 
72
- *Average of 3 independent benchmark runs.*
73
 
74
  | | Baseline (f16) | TurboQuant turbo3 | Delta |
75
  |--|:--------------:|:-----------------:|:-----:|
76
  | **Context** | 8,192 tokens | **64,000 tokens** | **+7.8×** |
77
  | **VRAM** | 5.7 GB | 6.2 GB | +0.54 GB only |
78
- | **Tokens/s** | 49.8 | 48.2 | **−3.2%** |
79
 
80
- > **7.8× more context. +0.5 GB VRAM. −3% speed. Even better ratio on smaller GPU.**
81
 
82
- Raw data: [`results/turboquant-4070-results-2026-04-01.json`](results/turboquant-4070-results-2026-04-01.json)
83
 
84
  ### Cross-GPU Summary
85
 
 
69
 
70
  ### RTX 4070 Laptop (8 GB) — Llama-3.1-8B-Instruct Q4_K_M
71
 
72
+ *2 independent benchmark sessions verified.*
73
 
74
  | | Baseline (f16) | TurboQuant turbo3 | Delta |
75
  |--|:--------------:|:-----------------:|:-----:|
76
  | **Context** | 8,192 tokens | **64,000 tokens** | **+7.8×** |
77
  | **VRAM** | 5.7 GB | 6.2 GB | +0.54 GB only |
78
+ | **Tokens/s (avg)** | 49.8 | 47.5 | **−4.6%** |
79
 
80
+ > **7.8× more context. +0.5 GB VRAM. −5% speed. Consistent across both runs.**
81
 
82
+ Raw data: [`results/turboquant-4070-results-2026-04-01.json`](results/turboquant-4070-results-2026-04-01.json) · [`results/turboquant-4070-laptop-2026-04-01.json`](results/turboquant-4070-laptop-2026-04-01.json)
83
 
84
  ### Cross-GPU Summary
85