aruunks66 commited on
Commit
3a2b17b
·
verified ·
1 Parent(s): a42daa7

Add Primitive organization card

Browse files

Brand banner, positioning, public Qwen3.8-27B metrics, models index. Contains no data from private repos.

.gitattributes CHANGED
@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ assets/org-banner.png filter=lfs diff=lfs merge=lfs -text
37
+ assets/qwen-metrics.png filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -1,10 +1,116 @@
1
  ---
2
  title: README
3
- emoji: 🦀
4
- colorFrom: green
5
- colorTo: blue
6
  sdk: static
7
  pinned: false
8
  ---
9
 
10
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  title: README
3
+ emoji: 🟧
4
+ colorFrom: yellow
5
+ colorTo: red
6
  sdk: static
7
  pinned: false
8
  ---
9
 
10
+ <p align="center">
11
+ <img src="assets/org-banner.png" alt="Primitive — inference economics for production LLM systems" width="100%">
12
+ </p>
13
+
14
+ <p align="center">
15
+ <img src="https://img.shields.io/badge/focus-inference%20economics-F98424?style=flat-square" alt="Inference economics">
16
+ <img src="https://img.shields.io/badge/runtimes-vLLM%20·%20ONNX-F98424?style=flat-square" alt="vLLM and ONNX">
17
+ <img src="https://img.shields.io/badge/hardware-Blackwell%20·%20A100%20·%20CPU-F98424?style=flat-square" alt="Blackwell, A100, CPU">
18
+ <img src="https://img.shields.io/badge/weights-Apache--2.0-lightgrey?style=flat-square" alt="Apache 2.0">
19
+ </p>
20
+
21
+ ---
22
+
23
+ ## Serving LLMs well is two problems
24
+
25
+ **Which model should answer this request?** and **how cheaply can that model
26
+ run?** Most teams solve neither — they pin one frontier model to every route and
27
+ pay frontier prices on the easy majority of their traffic.
28
+
29
+ We build the two pieces that fix that, and we publish the measurements.
30
+
31
+ <table>
32
+ <tr>
33
+ <td width="50%" valign="top">
34
+
35
+ ### 🔀 Route
36
+
37
+ A small, fast model that reads the prompt **before** you spend anything on it,
38
+ then ranks which LLM is actually worth calling. Runs on CPU next to your API
39
+ server — no GPU, no PyTorch, no extra service to operate.
40
+
41
+ *`llm-router-lite-80m-onnx` — private beta.*
42
+
43
+ </td>
44
+ <td width="50%" valign="top">
45
+
46
+ ### 🗜️ Compress
47
+
48
+ Mixed-precision quantization that shrinks a checkpoint and speeds up serving
49
+ **without quietly trading away accuracy** — and that still loads on the GPUs
50
+ you already have, not just the newest ones.
51
+
52
+ *`Qwen3.8-27B-mixed-NVFP4-FP8` — public, open weights.*
53
+
54
+ </td>
55
+ </tr>
56
+ </table>
57
+
58
+ ---
59
+
60
+ ## Latest release: Qwen3.8-27B, 2.5× smaller, faster, accuracy intact
61
+
62
+ <p align="center">
63
+ <img src="assets/qwen-metrics.png" alt="Qwen3.8-27B mixed NVFP4/FP8: 21 GiB vs 51.8 GiB BF16, accuracy 88.5 vs 88.6 BF16, 1134 tok/s vs 560 tok/s at concurrency 32" width="100%">
64
+ </p>
65
+
66
+ | config | size | accuracy | tok/s @ conc 32 | tok/s @ conc 1 |
67
+ |---|---|---|---|---|
68
+ | BF16 | 51.8 G | 88.6 | 559.6 | 26.3 |
69
+ | Qwen FP8 | 28.8 G | 87.6 | 884.0 | 45.6 |
70
+ | **primitive mixed NVFP4/FP8** | **~21 G** | **88.5** | **1134.5 (2.0×)** | **66.4 (2.5×)** |
71
+
72
+ Measured on a 1,170-sample mixed suite. Accuracy is statistically tied with BF16
73
+ (run-to-run spread ~±0.8). Serves on stock vLLM on **Blackwell** via native
74
+ FP4/FP8 kernels and on **Ampere/A100** via the Marlin W·A16 fallback — 88.1
75
+ accuracy, 790 tok/s @ concurrency 32 on an A100 40 GB. Speculative decoding
76
+ works on the quant: up to **+101%** with a DSpark drafter.
77
+
78
+ ```bash
79
+ vllm serve primitive-ai/Qwen3.8-27B-mixed-NVFP4-FP8 \
80
+ --reasoning-parser qwen3 \
81
+ --speculative-config '{"method":"mtp","num_speculative_tokens":2}'
82
+ ```
83
+
84
+ ---
85
+
86
+ ## How we publish
87
+
88
+ * **Numbers or nothing.** Every figure on our model cards comes from a held-out
89
+ suite and is reproducible from files in the repo. Where a metric is relative,
90
+ we say what it's relative to.
91
+ * **We document what breaks.** Our cards carry the failure modes, not just the
92
+ wins — which head dtype makes an A100 load succeed, which upstream config
93
+ field crashes vLLM, which optimisation we *rejected* and why.
94
+ * **No silent quality loss.** A compression or routing win that changes answers
95
+ isn't a win. When a cheaper option moved outputs too much, we don't ship it —
96
+ and we say so on the card.
97
+ * **Runs on hardware you have.** Ampere fallbacks for the quants, CPU-only for
98
+ the router. Nothing here assumes you just bought a Blackwell rack.
99
+
100
+ ---
101
+
102
+ ## Models
103
+
104
+ | repo | what it does | status |
105
+ |---|---|---|
106
+ | [`Qwen3.8-27B-mixed-NVFP4-FP8`](https://huggingface.co/primitive-ai/Qwen3.8-27B-mixed-NVFP4-FP8) | Mixed NVFP4/FP8 weights-only quant of Qwen3.8-27B. vLLM-native, Blackwell + A100, MTP head preserved for speculative decoding. | Public |
107
+ | `llm-router-lite-80m-onnx` | CPU-only router: reads a chat `messages` array, ranks which LLM to call. ONNX, no torch or transformers. | Private beta |
108
+
109
+ All published weights are Apache-2.0. Derivative models carry upstream
110
+ attribution and a `NOTICE` file.
111
+
112
+ <p align="center">
113
+ <br>
114
+ <img src="assets/primitive-logo.png" alt="Primitive" width="34"><br>
115
+ <sub><b>primitive</b> · inference economics for production LLM systems</sub>
116
+ </p>
assets/org-banner.png ADDED

Git LFS Details

  • SHA256: c735727268d467b5c75d6ae182f9c29025435924f0ad52cef231484d4b1f5a52
  • Pointer size: 131 Bytes
  • Size of remote file: 122 kB
assets/primitive-logo.png ADDED
assets/qwen-metrics.png ADDED

Git LFS Details

  • SHA256: 7fad8eb0bc6357c8f06c390406b9bd41a2ae9e357ef90178eadb85d58eaae80d
  • Pointer size: 131 Bytes
  • Size of remote file: 143 kB