Why your GPU's TFLOPs do not decide your tokens per second.
At batch one, a language model reads every weight from memory to produce one token, and uses each weight exactly once. Arithmetic intensity is about 0.5 FLOP per byte. The accelerator's FLOP rating never enters the equation — the only lever on single-stream speed is how many bytes the model is.
| Format | Bits/weight | Size | Fits | Decode ceiling |
|---|
This is a ceiling, not a promise. Attention, the KV cache, kernel overhead and real bandwidth utilisation (typically 70–85% of the spec sheet) all put the achievable number below it. Larger batches raise arithmetic intensity and change the picture entirely — this models the single-stream case, which is what most people run locally. Bandwidth figures are vendor peak.
Bytes are the only lever at batch one, so these are what the lever is worth in practice. Accuracy retained is the mean of ARC-Easy, ARC-Challenge, HellaSwag and PIQA against the uncompressed model, scored on the same GPU and harness. The speech model has no such benchmark, so it is measured by word error rate instead.
| Model | Base | Size | Smaller by | Accuracy retained |
|---|---|---|---|---|
| Exeaon1-Nunya-14B | Qwen3-14B | 7.38 GB | 3.73x | 100.3% |
| Exeaon1-Kese-30B-A3B | Qwen3-30B-A3B | 16.27 GB | 3.50x | 98.1% |
| Exeaon1-Nunya-8B | Qwen3-8B | 4.11 GB | 3.80x | 99.6% |
| Exeaon1-Dzo-4B | Qwen3-4B | 2.04 GB | 3.67x | 98.1% |
| Exeaon1-Dzo-0.6B | Qwen3-0.6B | 0.31 GB | 3.50x | 96.3% |
| Exeaon1-Voice-0.8B | Whisper v3-turbo | 0.43 GB | 3.47x | WER unchanged |
Retention is not a free lunch, and neither is speed. On a large GPU these run slower than dense fp16 — a vendor tensor-core GEMM is heavily tuned and there is bandwidth to spare, so trading compute for memory loses. The win is on the other side of this page: fitting where the dense model does not.
Compressed models: huggingface.co/Exeaon · Runtime: github.com/ExeaonLM · Zenux Plimver Technologies LTD, Ghana