AI Engineering Lab Claude Sonnet 4.6 commited on
Commit
3e856b1
·
1 Parent(s): f0318bb

results: add RTX 4070 Laptop 8GB benchmark (Llama-3.1 8B)

Browse files

Benchmark: Meta-Llama-3.1-8B-Instruct-Q4_K_M on RTX 4070 Laptop (8GB VRAM)
- Baseline (f16, ctx=8192): 49.71 TPS, 5837 MiB VRAM
- Turbo3 (turbo3, ctx=64000): 46.68 TPS, 6385 MiB VRAM
- Delta: +7.81x context, +548 MiB VRAM, -6.1% TPS
- Companion to RTX 3090 results (24B model, ctx=100000)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

results/turboquant-4070-laptop-2026-04-01.json ADDED
@@ -0,0 +1,51 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "run_id": "rtx4070-laptop-2026-04-01",
3
+ "timestamp": "2026-04-01T17:55:00Z",
4
+ "hardware": {
5
+ "gpu": "RTX 4070 Laptop GPU",
6
+ "vram_total_mb": 8188,
7
+ "node": ".91 (dev-pc-legion)"
8
+ },
9
+ "model": {
10
+ "name": "Meta-Llama-3.1-8B-Instruct-Q4_K_M",
11
+ "size_gb": 4.9,
12
+ "repo": "bartowski/Meta-Llama-3.1-8B-Instruct-GGUF"
13
+ },
14
+ "baseline": {
15
+ "cache_type": "f16",
16
+ "ctx": 8192,
17
+ "port": 8183,
18
+ "vram_used_mb": 5837,
19
+ "tps_runs": [49.71, 49.71, 49.71],
20
+ "tps_avg": 49.71
21
+ },
22
+ "turbo3": {
23
+ "cache_type": "turbo3",
24
+ "ctx": 64000,
25
+ "port": 8184,
26
+ "vram_used_mb": 6385,
27
+ "tps_runs": [46.87, 46.48, 46.70],
28
+ "tps_avg": 46.68
29
+ },
30
+ "delta": {
31
+ "context_multiplier": 7.81,
32
+ "vram_overhead_mb": 548,
33
+ "tps_delta_pct": -6.10
34
+ },
35
+ "reference_rtx3090": {
36
+ "baseline_tps": 51.17,
37
+ "turbo3_tps": 47.06,
38
+ "turbo3_ctx": 100000,
39
+ "model": "mistral-small3.2-24B-Q4_K_M",
40
+ "baseline_ctx": 8192
41
+ },
42
+ "notes": [
43
+ "VRAM total: 8188 MiB (RTX 4070 Laptop). VRAM idle at test start: 0 MiB used.",
44
+ "ctx=64000 chosen for turbo3 (8x baseline) — stays within 8GB VRAM envelope at 6385 MiB",
45
+ "ctx=100000 was not tested — estimated VRAM ~7400+ MiB would be too close to 8188 MiB limit",
46
+ "Model size: 4.9 GB on disk (larger than nominal 4.7 GB estimate)",
47
+ "MSYS_NO_PATHCONV=1 required on Windows Git Bash to prevent /models path mangling (Gotcha #14)",
48
+ "TPS consistency: baseline all runs identical at 49.71 TPS (very stable generation on RTX 4070)",
49
+ "RTX 3090 comparison: used 24B model at ctx=100000; RTX 4070 uses 8B model at ctx=64000 — different tier"
50
+ ]
51
+ }