--- title: README emoji: 🟧 colorFrom: yellow colorTo: red sdk: static pinned: false ---

--- ## Serving LLMs well is two problems **Which model should answer this request?** and **how cheaply can that model run?** Most teams solve neither — they pin one frontier model to every route and pay frontier prices on the easy majority of their traffic. We build the two pieces that fix that.

### 🔀 Route A small, fast model that reads the prompt **before** you spend anything on it, then ranks which LLM is actually worth calling. Runs on CPU next to your API server — no GPU, no PyTorch, no extra service to operate.

### 🗜️ Compress Mixed-precision quantization that shrinks a checkpoint and speeds up serving **without quietly trading away accuracy** — and that still loads on the GPUs you already have, not just the newest ones.

--- ## Out now: six open-weight quants

Across the six public quants: 2.5× to 3.4× smaller than BF16; 1.72× to 2.68× BF16 throughput at concurrency 32; accuracy within 0.5 points of BF16 in either direction, inside the ±0.8 run-to-run band — ties, not wins

Weights-only mixed precision, `compressed-tensors`, stock `vllm serve`. Every figure below is on the model's own card, next to the run that produced it. * 🧮 **[Qwen3.8-27B — mixed NVFP4/FP8](https://huggingface.co/primitive-ai/Qwen3.8-27B-mixed-NVFP4-FP8)** · 20.7 GiB, **2.5× smaller than BF16** and 28% smaller than Qwen's own FP8, with accuracy tied to BF16 (88.5 vs 88.6 — and the same weights re-scored 87.5 in a second sitting, which is what a tie looks like). **Loads on an A100**, not just Blackwell, and the in-checkpoint MTP head survives, so speculative decoding still works. * 🪶 **[Laguna-XS-2.1 — mixed NVFP4/MXFP8](https://huggingface.co/primitive-ai/Laguna-XS-2.1-mixed-NVFP4-MXFP8)** · 19.3 GiB at 4.96 bits/weight, **3.2× smaller than BF16 and smaller than every official quant** — while running faster than all of them at 3,268 tok/s. * 🐘 **[Laguna-S-2.1 — mixed NVFP4/MXFP8](https://huggingface.co/primitive-ai/Laguna-S-2.1-mixed-NVFP4-MXFP8)** · 219 GB of source weights down to 64 GiB at 4.67 bits/weight — **it serves from one 96 GiB GPU**, the smallest and fastest of the 4-bit field, with +23% single-stream decode over the NVFP4 alternative at a 3 GiB smaller footprint. * 👁️ **[Muse-Glimmer-30B — mixed NVFP4](https://huggingface.co/primitive-ai/Muse-Glimmer-30B-mixed-NVFP4)** · A vision-language model at 20.4 GiB, **2.40× BF16 throughput** and 86.0 against BF16's 86.2. **The vision tower stays BF16** — a VLM loses its visual grounding before it loses its words. * 🦅 **[Ornith-1.5-35B-A3B — mixed NVFP4/FP8](https://huggingface.co/primitive-ai/Ornith-1.5-35B-A3B-mixed-NVFP4-FP8)** · A 35 B MoE vision-language model at 22.6 GiB, **3.0× smaller than BF16** and 39% smaller than the official FP8, scoring 91.5 against BF16's 92.0 — the only quant of this model we have measured level with BF16. **The official NVFP4 is faster and 0.75 GiB smaller**; this build trades that for the closest accuracy to BF16 and a BF16 output head that keeps the Ampere path loadable. * ⚡ **[Nemotron-3.5-Lightning-30B-A3B — mixed INT4/INT8](https://huggingface.co/primitive-ai/Nemotron-3.5-Lightning-30B-A3B-mixed-INT4-INT8)** · A MoE+Mamba hybrid at 19.2 GiB and 5.01 bits/weight, **3.3× smaller than BF16 at 2.7× its throughput** — and **8.6% smaller and ~7.7% faster than nvidia's own NVFP4**, from weights-only round-to-nearest with no calibration data. We make no accuracy claim here: identical weights scored 88.0 and 87.4 on two runs, so that column is a tie. --- ## Still in private beta The router is not published yet. We're working with a small number of teams running real production traffic — the ones who can tell us where the sharp edges are before everyone else finds them. **A good fit if you're:** paying per-token to a frontier API on traffic you suspect doesn't need it, serving an open-weights model at meaningful volume, or squeezing a large checkpoint onto GPUs you already own.

Request access at primitive.com →

--- ## How we publish * **Numbers or nothing.** Every figure on our model cards comes from a held-out suite and is reproducible from files in the repo. Where a metric is relative, we say what it's relative to. * **We document what breaks.** Our cards carry the failure modes, not just the wins — the dtype choices that matter, the upstream quirks that bite, and the optimisations we tried and *rejected*. * **No silent quality loss.** A compression or routing win that changes answers isn't a win. When a cheaper option moved outputs too much, we don't ship it — and we say so on the card. * **A tie is a tie.** Our accuracy deltas sit inside the run-to-run spread of the suite, and we label them as ties rather than dressing them up as wins. * **Runs on hardware you have.** Older-generation GPU fallbacks for the quants, CPU-only for the router. Nothing here assumes you just bought the newest rack. --- ## Repositories | repo | what it does | status | |---|---|---| | `Qwen3.8-27B-mixed-NVFP4-FP8` | Mixed-precision quant of Qwen3.8-27B. vLLM-native, A100 fallback, MTP head preserved for speculative decoding. | **Open weights** | | `Laguna-XS-2.1-mixed-NVFP4-MXFP8` | Mixed-precision quant of poolside's Laguna-XS-2.1. Smallest and fastest build of it we've measured. | **Open weights** | | `Laguna-S-2.1-mixed-NVFP4-MXFP8` | Mixed-precision quant of poolside's Laguna-S-2.1. 219 GB of weights onto a single 96 GiB card. | **Open weights** | | `Muse-Glimmer-30B-mixed-NVFP4` | NVFP4 quant of the Muse-Glimmer-30B VLM, GPTQ-calibrated. Vision tower left at BF16. | **Open weights** | | `Ornith-1.5-35B-A3B-mixed-NVFP4-FP8` | Mixed-precision quant of Ornith-1.5-35B-A3B, a 35 B MoE VLM. Vision tower and MTP block left at BF16. | **Open weights** | | `Nemotron-3.5-Lightning-30B-A3B-mixed-INT4-INT8` | INT4/INT8 quant of nvidia's MoE+Mamba hybrid. Smaller and faster than the official NVFP4, no calibration data. | **Open weights** | | `llm-router-lite-80m-onnx` | CPU-only router: reads a chat `messages` array, ranks which LLM to call. ONNX, no torch or transformers. | Private beta | Published weights carry their upstream licence — Apache-2.0 for the Qwen and Muse-Glimmer builds, OpenMDW-1.1 for the Laguna and Nemotron builds, MIT for the Ornith build. Derivative models carry upstream attribution and a `NOTICE` or `LICENSE` file. ---

Cutting your inference bill?

We'd like to hear what you're serving and where it hurts.
primitive.com

_{primitive · inference economics for production LLM systems}