Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -17,7 +17,7 @@ NVFP4 (4-bit) quant of [CohereLabs/North-Mini-Code-1.0](https://huggingface.co/C
|
|
| 17 |
|
| 18 |
**Benchmarks (DGX Spark / GB10, vLLM):**
|
| 19 |
- HumanEval pass@1: **90.2%** — matches the FP8 build (90.2%), within one problem of bf16 (90.9%).
|
| 20 |
-
- Decode: **~
|
| 21 |
|
| 22 |
Serve with vLLM (needs `Cohere2MoeForCausalLM` support + `cohere_melody`):
|
| 23 |
|
|
|
|
| 17 |
|
| 18 |
**Benchmarks (DGX Spark / GB10, vLLM):**
|
| 19 |
- HumanEval pass@1: **90.2%** — matches the FP8 build (90.2%), within one problem of bf16 (90.9%).
|
| 20 |
+
- Decode: **~58 tok/s** single-request, reasoning off (spark-arena nightly image) — **≈1.67×** the FP8 build, at ~17 GB vs ~28 GB.
|
| 21 |
|
| 22 |
Serve with vLLM (needs `Cohere2MoeForCausalLM` support + `cohere_melody`):
|
| 23 |
|