Nicolassuez/Qwen3-0.6B-eagle3

EAGLE-3 draft head for Qwen3-0.6B (tool-calling tuned)

A speculative-decoding draft head trained with SpecForge for Qwen/Qwen3-0.6B, aimed at fast single-user JSON tool calling. Qwen ships MTP/NEXTN heads only for its larger models, so nothing existed for this size: this head was trained in-house.

Current revision = v3, trained in three stages on an 85k-conversation mix (40k glaive-function-calling-v2 + 30k ultrachat + 15k synthetic water/IoT tool calls whose argument values are BOUNDED: enums, structured ids, fixed units, which is what production tool schemas look like). The last stage adds 42k steps at ttt_length 7 and max_length 4096, the two settings the earlier revisions had to give up on a shared GPU.

Measured (H100 PCIe, quiet card, SGLang 2026-07-20 nightly, fa3, 500-token decode)

Config	Single-stream
Qwen3-0.6B, no speculation	553 tok/s
+ this head, v1 (40k steps)	814 tok/s
+ this head, v2	861 tok/s
+ this head, v3 (current)	955 tok/s (+73%)

By generation domain, v3 versus v2 under identical conditions: generic 924 vs 650, free-form JSON 782 vs 566 (+38%), bounded-argument domain JSON 757 vs 592 (+28%). The domain dataset pays where argument values are enums and ids rather than free text, which is the production case.

Speculation config sweep: steps 3 / topk 1 / draft 4 is the optimum. Deeper or wider LOSES (steps 4: 705, steps 5: 631, topk 2: 757 on v1): verification cost outgrows acceptance on a 0.6B target. ngram speculation is a net regression at batch 1 (426-445 versus 665 baseline).

Usage (SGLang)

python -m sglang.launch_server \
  --model-path Qwen/Qwen3-0.6B \
  --speculative-algo EAGLE3 \
  --speculative-draft-model-path Nicolassuez/Qwen3-0.6B-eagle3 \
  --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 \
  --attention-backend fa3

Serving footprint measured at ~3.5 GiB (context 4096).

Training notes

• Draft: 1-layer llama-style EAGLE-3 (hidden 1024, draft vocab 32k), auxiliary hidden states captured from target layers 1/13/24.
• ttt_length 7, max_length 4096, batch size 1, lr 1e-4. Earlier revisions used ttt 5 / 2048 because the training GPU was shared with production services; lifting both is what took 861 to 955.
• Trained in completable chunks, each warm-started from the previous one.
• The vocabulary mapping (d2t/t2d) is baked in; any continuation must reuse the same mapping.