---
license: cc-by-nc-4.0
base_model: CohereLabs/North-Mini-Code-1.0
tags:
- eagle3
- speculative-decoding
- cohere2_moe
---

# North-Mini-Code-1.0 — EAGLE-3 draft head

EAGLE-3 draft model for `CohereLabs/North-Mini-Code-1.0` (`cohere2_moe`, 30B/3B MoE, 49 layers),
trained with SpecForge (offline) for lossless speculative decoding.

- **Draft**: 1 Llama-style decoder layer, hidden 2048, FFN 12288, draft_vocab 32000 (freq-reduced from 262144).
- **Aux hidden-state layers**: [1, 23, 45].
- **Training**: offline, ~8.3k code-instruction samples (magicoder-evol-instruct), 10 epochs, lr 1e-4.
- **Offline held-out acceptance** (Σ over 7 positions): τ = 4.25 (pos-0 acc 0.71).

## Serving in vLLM
Needs vLLM main + `--hf-overrides '{"first_k_dense_replace":1}'`, and a patch adding the EAGLE3
interface (`SupportsEagle3`) to `cohere2_moe.py`. See repo notes.

```bash
vllm serve CohereLabs/North-Mini-Code-1.0 \
  --speculative-config '{"method":"eagle3","model":"<this-repo>","num_speculative_tokens":5}' \
  --hf-overrides '{"first_k_dense_replace":1}' \
  --reasoning-parser cohere_command4 --tool-call-parser cohere_command4 --enable-auto-tool-choice
```

> Note: this draft was trained **offline** on HuggingFace-transformers hidden states; real vLLM
> acceptance is modest (~1.28) due to train/serve hidden-state representation mismatch. For best
> speedup, retrain **online** in vLLM/SpecForge so the draft matches serving-time hidden states.