---
license: apache-2.0
base_model: CohereLabs/North-Mini-Code-1.0
base_model_relation: quantized
pipeline_tag: text-generation
tags:
  - nvfp4
  - compressed-tensors
  - vllm
  - moe
  - code
---

# North-Mini-Code-1.0-NVFP4

NVFP4 (4-bit) quant of [CohereLabs/North-Mini-Code-1.0](https://huggingface.co/CohereLabs/North-Mini-Code-1.0), made with [llm-compressor](https://github.com/vllm-project/llm-compressor) (`nvfp4-pack-quantized`). ~17 GB. Experts quantized; router/gate/`lm_head` kept high-precision.

**Benchmarks (DGX Spark / GB10, vLLM):**
- HumanEval pass@1: **90.2%** — matches the FP8 build (90.2%), within one problem of bf16 (90.9%).
- Decode: **~58 tok/s** single-request, reasoning off (spark-arena nightly image) — **≈1.67×** the FP8 build, at ~17 GB vs ~28 GB.

Serve with vLLM (needs `Cohere2MoeForCausalLM` support + `cohere_melody`):

```bash
vllm serve XanuNetworks/North-Mini-Code-1.0-NVFP4 \
  --max-model-len 262144 \
  --enable-auto-tool-choice \
  --tool-call-parser cohere_command4 \
  --reasoning-parser cohere_command4
```

Apache-2.0, inherited from the base model. All credit to Cohere for North Mini Code.