AI Engineering Lab commited on
Commit
78e199b
·
1 Parent(s): 3e856b1

results: add verified RTX 4070 Laptop benchmark + cross-GPU comparison table

Browse files

- RTX 4070 Laptop (8GB): 8,192 -> 64,000 ctx (+7.8x), -3.2% TPS, +0.54 GB VRAM
- 3 independent runs verified on Llama-3.1-8B-Instruct Q4_K_M
- Add cross-GPU comparison table (3090 vs 4070) to README
- Update HF Space frontmatter title + tags for both GPUs

README.md CHANGED
@@ -1,5 +1,5 @@
1
  ---
2
- title: TurboQuant on RTX 3090 — 100K Context, 4.3x KV-Cache Compression
3
  emoji: 🚀
4
  colorFrom: blue
5
  colorTo: purple
@@ -13,6 +13,7 @@ tags:
13
  - turboquant
14
  - benchmark
15
  - rtx3090
 
16
  - consumer-hardware
17
  - mistral
18
  - llama-cpp
@@ -46,8 +47,11 @@ tags:
46
 
47
  ## 📊 Results
48
 
49
- Tested on **NVIDIA RTX 3090 (24 GB VRAM)** with **Mistral-Small-3.2-24B Q4_K_M**.
50
- Results are the average of **two independent benchmark runs** (April 1, 2026).
 
 
 
51
 
52
  | | Baseline (f16) | TurboQuant turbo3 | Delta |
53
  |--|:--------------:|:-----------------:|:-----:|
@@ -58,11 +62,34 @@ Results are the average of **two independent benchmark runs** (April 1, 2026).
58
 
59
  > **12× more context. +12% VRAM. −8% speed. Same model weights.**
60
 
61
- **Run 1 (cold):** Baseline 49.2 TPS / 15,408 MB → Turbo3 45.0 TPS / 17,224 MB
62
- **Run 2 (warm, idle GPU):** Baseline 51.2 TPS / 15,695 MB → Turbo3 47.1 TPS / 17,581 MB
63
 
64
  Raw data: [`results/turboquant-rtx3090-2026-04-01.json`](results/turboquant-rtx3090-2026-04-01.json) · [`results/turboquant-rtx3090-2026-04-01-v2.json`](results/turboquant-rtx3090-2026-04-01-v2.json)
65
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
66
  ---
67
 
68
  ## 🚀 Quick Start
 
1
  ---
2
+ title: TurboQuant on Consumer GPUs — 100K Context on RTX 3090, 64K on RTX 4070
3
  emoji: 🚀
4
  colorFrom: blue
5
  colorTo: purple
 
13
  - turboquant
14
  - benchmark
15
  - rtx3090
16
+ - rtx4070
17
  - consumer-hardware
18
  - mistral
19
  - llama-cpp
 
47
 
48
  ## 📊 Results
49
 
50
+ Tested on two consumer GPUs. Results verified across multiple independent runs (April 1, 2026).
51
+
52
+ ### RTX 3090 (24 GB) — Mistral-Small-3.2-24B Q4_K_M
53
+
54
+ *Average of 2 independent benchmark runs.*
55
 
56
  | | Baseline (f16) | TurboQuant turbo3 | Delta |
57
  |--|:--------------:|:-----------------:|:-----:|
 
62
 
63
  > **12× more context. +12% VRAM. −8% speed. Same model weights.**
64
 
65
+ Run 1 (cold): Baseline 49.2 TPS / 15,408 MB → Turbo3 45.0 TPS / 17,224 MB
66
+ Run 2 (warm): Baseline 51.2 TPS / 15,695 MB → Turbo3 47.1 TPS / 17,581 MB
67
 
68
  Raw data: [`results/turboquant-rtx3090-2026-04-01.json`](results/turboquant-rtx3090-2026-04-01.json) · [`results/turboquant-rtx3090-2026-04-01-v2.json`](results/turboquant-rtx3090-2026-04-01-v2.json)
69
 
70
+ ### RTX 4070 Laptop (8 GB) — Llama-3.1-8B-Instruct Q4_K_M
71
+
72
+ *Average of 3 independent benchmark runs.*
73
+
74
+ | | Baseline (f16) | TurboQuant turbo3 | Delta |
75
+ |--|:--------------:|:-----------------:|:-----:|
76
+ | **Context** | 8,192 tokens | **64,000 tokens** | **+7.8×** |
77
+ | **VRAM** | 5.7 GB | 6.2 GB | +0.54 GB only |
78
+ | **Tokens/s** | 49.8 | 48.2 | **−3.2%** |
79
+
80
+ > **7.8× more context. +0.5 GB VRAM. −3% speed. Even better ratio on smaller GPU.**
81
+
82
+ Raw data: [`results/turboquant-4070-results-2026-04-01.json`](results/turboquant-4070-results-2026-04-01.json)
83
+
84
+ ### Cross-GPU Summary
85
+
86
+ | GPU | VRAM | Model | Max Context (turbo3) | Speed Loss |
87
+ |-----|------|-------|---------------------|-----------|
88
+ | RTX 3090 | 24 GB | Mistral-Small-3.2 24B | 100,000 tokens | −8.3% |
89
+ | RTX 4070 Laptop | 8 GB | Llama-3.1 8B | 64,000 tokens | −3.2% |
90
+
91
+ TurboQuant scales with the GPU: the principle (+7-12× context, minimal speed loss) holds across hardware classes.
92
+
93
  ---
94
 
95
  ## 🚀 Quick Start
results/turboquant-4070-results-2026-04-01.json ADDED
@@ -0,0 +1,38 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "date": "2026-04-01",
3
+ "hardware": {
4
+ "gpu": "NVIDIA GeForce RTX 4070 Laptop GPU",
5
+ "vram_gb": 8,
6
+ "node": ".91 (dev-pc-legion)"
7
+ },
8
+ "model": "Meta-Llama-3.1-8B-Instruct-Q4_K_M",
9
+ "model_size_gb": 4.7,
10
+ "runs": 3,
11
+ "baseline": {
12
+ "ctx": 8192,
13
+ "kv_type": "f16",
14
+ "vram_mb": 5835,
15
+ "tps_runs": [
16
+ 50.01,
17
+ 49.87,
18
+ 49.61
19
+ ],
20
+ "tps_avg": 49.83
21
+ },
22
+ "turboquant": {
23
+ "ctx": 64000,
24
+ "kv_type": "turbo3",
25
+ "vram_mb": 6383,
26
+ "tps_runs": [
27
+ 48.21,
28
+ 48.26,
29
+ 48.22
30
+ ],
31
+ "tps_avg": 48.23
32
+ },
33
+ "delta": {
34
+ "ctx_multiplier": 7.8,
35
+ "vram_delta_gb": 0.54,
36
+ "tps_delta_pct": -3.2
37
+ }
38
+ }