AI Engineering Lab commited on
Commit
571e9fb
·
1 Parent(s): 87efc66

results: add v2 verification run — confirms v1 within measurement variance

Browse files
README.md CHANGED
@@ -23,17 +23,21 @@
23
  ## 📊 Results
24
 
25
  Tested on **NVIDIA RTX 3090 (24 GB VRAM)** with **Mistral-Small-3.2-24B Q4_K_M**.
 
26
 
27
  | | Baseline (f16) | TurboQuant turbo3 | Delta |
28
  |--|:--------------:|:-----------------:|:-----:|
29
  | **Context** | 8,192 tokens | **100,000 tokens** | **+12.2×** |
30
- | **VRAM** | 15.4 GB | 17.2 GB | +1.8 GB only |
31
- | **Tokens/s** | 49.2 | 45.0 | −8.5% |
32
  | **KV-Cache size** | ~1 GB (f16) | ~2.8 GB (3-bit) | **4.3× compression** |
33
 
34
- > **12× more context. +11% VRAM. −8% speed. Same model weights.**
35
 
36
- Raw data: [`results/turboquant-rtx3090-2026-04-01.json`](results/turboquant-rtx3090-2026-04-01.json)
 
 
 
37
 
38
  ---
39
 
 
23
  ## 📊 Results
24
 
25
  Tested on **NVIDIA RTX 3090 (24 GB VRAM)** with **Mistral-Small-3.2-24B Q4_K_M**.
26
+ Results are the average of **two independent benchmark runs** (April 1, 2026).
27
 
28
  | | Baseline (f16) | TurboQuant turbo3 | Delta |
29
  |--|:--------------:|:-----------------:|:-----:|
30
  | **Context** | 8,192 tokens | **100,000 tokens** | **+12.2×** |
31
+ | **VRAM** | 15.5 GB | 17.4 GB | +1.9 GB only |
32
+ | **Tokens/s** | 50.2 | 46.0 | **−8.3%** |
33
  | **KV-Cache size** | ~1 GB (f16) | ~2.8 GB (3-bit) | **4.3× compression** |
34
 
35
+ > **12× more context. +12% VRAM. −8% speed. Same model weights.**
36
 
37
+ **Run 1 (cold):** Baseline 49.2 TPS / 15,408 MB → Turbo3 45.0 TPS / 17,224 MB
38
+ **Run 2 (warm, idle GPU):** Baseline 51.2 TPS / 15,695 MB → Turbo3 47.1 TPS / 17,581 MB
39
+
40
+ Raw data: [`results/turboquant-rtx3090-2026-04-01.json`](results/turboquant-rtx3090-2026-04-01.json) · [`results/turboquant-rtx3090-2026-04-01-v2.json`](results/turboquant-rtx3090-2026-04-01-v2.json)
41
 
42
  ---
43
 
results/turboquant-rtx3090-2026-04-01-v2.json ADDED
@@ -0,0 +1,36 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "run_id": "v2-verification-2026-04-01",
3
+ "timestamp": "2026-04-01T00:00:00Z",
4
+ "hardware": {
5
+ "gpu": "RTX 3090",
6
+ "vram_total_mb": 24576
7
+ },
8
+ "baseline": {
9
+ "cache_type": "f16",
10
+ "ctx": 8192,
11
+ "port": 8180,
12
+ "vram_used_mb": 15695,
13
+ "tps_runs": [51.01, 51.07, 51.44],
14
+ "tps_avg": 51.17
15
+ },
16
+ "turbo3": {
17
+ "cache_type": "turbo3",
18
+ "ctx": 100000,
19
+ "port": 8182,
20
+ "vram_used_mb": 17581,
21
+ "tps_runs": [47.34, 46.90, 46.93],
22
+ "tps_avg": 47.06
23
+ },
24
+ "delta": {
25
+ "context_multiplier": 12.21,
26
+ "vram_overhead_mb": 1886,
27
+ "tps_delta_pct": -8.03
28
+ },
29
+ "model": "mistralai_Mistral-Small-3.2-24B-Instruct-2506-Q4_K_M.gguf",
30
+ "image": "turboquant-plus:feature",
31
+ "notes": [
32
+ "VRAM was nearly idle at start (550 MB used, 23777 MB free) — no unloading needed",
33
+ "Port 8182 used for turbo3 (8181 occupied by ocr-api-waitress)",
34
+ "v2 verification confirms v1 results: turbo3 enables 12x context at +1.9 GB VRAM cost and -8% TPS"
35
+ ]
36
+ }