VKUE: No GPU? Runs Anyway — a 34.7B Reasoner on a Laptop and on Bare CPU
TL;DR
- We ran a single 34.7B reasoning model (Ourbox-35B-JGOS) as the exact same weights from a datacenter B200, down to an 8 GB gaming laptop, down to a CPU server with no GPU at all.
- Measured: B200 18,057 tok/s (aggregate) → A10G 126 tok/s → 8 GB laptop 20 tok/s → GPU-less CPU 17 tok/s. One file, the whole way.
- The trick isn't a new kernel — it's the model structure. Of the 34.7B parameters, only ~3B are active per token (sparse MoE), so per-token cost collapses to a 3B-class budget.
- Try it yourself — the live demos below stream the same prompt on GPU and CPU side by side, with a live tok/s counter.
The assumption: "big model = big hardware"
You need a datacenter GPU to run a frontier-class model — mostly true. But that assumption rests on another one: that the model reads all of its parameters on every token. Decoding is bound by memory bandwidth, not compute. If you have to sweep 34 billion parameters per token, an 8 GB card collapses to one or two tokens per second.
That assumption isn't always true.
VKUE — an engine for ubiquity
VIDRAFT's serving line has two branches.
| Goal | Slogan | |
|---|---|---|
| VKAE | Speed (max datacenter throughput) | fast |
| VKUE | Ubiquity (run on minimal hardware) | anywhere |
VKUE (VIDRAFT Kernel Ubiquitous Engine) is about accessibility, not raw speed: taking a large, capable model and making it run on hardware where it normally couldn't — a GPU-less CPU box, or an 8 GB laptop.
This post demonstrates that claim with a single model.
Why a 34.7B model runs on a CPU — the physics of active parameters
The model is Ourbox-35B-JGOS, a sparse Mixture-of-Experts reasoner from the Qwen3.5-MoE / Qwen3-Next family (34.7B total, 256 experts top-8, Gated-DeltaNet linear attention interleaved with full attention).
One fact drives everything:
The model holds 34.7B parameters, but only about 3B are active per token.
Because decoding is memory-bandwidth bound, what matters is how many bytes move per token:
- Dense 34B: ~16.7 GB moved per token → collapses on an 8 GB card.
- Ourbox (A3B): ~1.45 GB moved per token → about 11× less memory traffic.
Keep the experts in system RAM and only the attention/router/shared layers on the GPU, and a 34.7B-class reasoner runs at usable speed on an 8 GB laptop — or with no GPU at all.
Same weights, the whole spectrum (all measured)
The key table. One file; only the hardware changes.
| Tier | Hardware | Throughput | Serving |
|---|---|---|---|
| Datacenter ceiling | single B200 | 18,057 tok/s (aggregate) | VIDRAFT optimized serving (VKAE) |
| 1× cloud GPU | single A10G | 126 tok/s (single-stream) | VKUE, open GGUF |
| Consumer floor | 8 GB laptop (RTX 5060) | 20.01 tok/s | VKUE, open GGUF |
| No GPU | CPU-only server | ~17 tok/s | VKUE, open GGUF |
Every number is measured. From a datacenter B200 down to a gaming laptop, one set of weights spans four orders of magnitude of hardware.
Sparsity alone — an honest A/B
To rule out "it's just a small model," we ran a head-to-head against a dense model on the same laptop, same engine, same quantization (Q3_K_M), near-identical footprint.
| Model | Active params | Footprint | Decode (same laptop) | Basis |
|---|---|---|---|---|
| Ourbox-35B (A3B), this repo | ~3B | 15.6 GiB | 20.01 tok/s | measured |
| Qwen2.5-32B (dense) | 32.8B | 14.84 GiB | 5.36 tok/s | measured (our A/B) |
→ 3.7× faster from sparsity alone. The only variable is active parameters (3B vs 32.8B). That 20 tok/s is also ~2× the best documented dense-32B result on any 8 GB machine (~10.8).
And it's not a toy — Ourbox-35B posts GPQA Diamond 86.4% (maj@8) / 70.7% (greedy) (measured, conditions labeled).
Try it — live demos
Numbers are hollow without proof, so measure it live. Enter a prompt and watch two hardware paths generate side by side, tok/s ticking up in real time. Being a reasoning model, each panel streams its private thinking (dimmed) and then the answer.
- 🟢🔵 GPU vs CPU — one box, two panels The GPU path and a GPU-less CPU path on the same A10G box. The only variable is the hardware.
- 🔵 CPU only — no GPU A pure CPU server, no accelerator at all. Slow — but it runs. That's the point.
Honest caveats
Honesty is the whole point of a benchmark, so, explicitly:
- These are one machine's live measurements, not universal claims.
- The CPU path proves the model runs without a GPU — it is not claimed to beat a GPU.
- Weights are quantized (Q3_K_M) so one file fits every tier. Held entirely in a 24 GB card's VRAM, an A3B reaches 87–196 tok/s; the 8 GB / CPU numbers above are explicitly the bottom tier.
- The consumer-tier numbers are reproducible with open tooling — the exact run instructions are public in the model card.
Why it matters
Sovereign on-prem, edge, and public-sector deployments often can't touch the cloud or an H100. VKUE puts a frontier-class reasoner in those places — on a single $1,600 card, a gaming laptop, or a CPU server you already own. If you have a big card it's fast; if you don't, it still runs. Anywhere.
Links
🔵 GPU vs CPU demo: https://final-bench-ourbox-35b-vkue-demo.hf.space
🔵 CPU-only demo: https://final-bench-ourbox-35b-vkue-cpu.hf.space
📊 VKUE leaderboard: https://huggingface.co/spaces/FINAL-Bench/VKUE
🤗 Model: https://huggingface.co/FINAL-Bench/Ourbox-35B-JGOS-GGUF
⚡ VKAE (speed): https://huggingface.co/spaces/VIDraft/vkae
VKUE is part of VIDRAFT's efficiency-serving line — the same weights run from a single datacenter GPU down to a consumer laptop. "VKAE is fast; VKUE is everywhere."
