README.md

---
license: apache-2.0
language:
- en
tags:
- code
- knowledge-graph
- labeling
- summarization
- distillation
- qwen3
- gguf
- llama.cpp
pipeline_tag: text-generation
base_model: Qwen/Qwen3-0.6B
library_name: gguf
---

# code-daemon-enrich-v1

A distilled **Qwen3-0.6B** worker that writes the **short, structured labels** in the UltraCode
code-intelligence pipeline: RAPTOR L0/L1 cluster labels, community labels, and link-selection
picks. It replaces a 7B teacher on exactly the high-volume, **prefill-bound, short-output** stages
where a sub-billion-parameter model is enough — running those stages on the daemon's dedicated
`.enrich` worker at a fraction of the main LLM's per-call prefill cost.

This is a **purpose-built pipeline component, not a general assistant.** It only does the four
label tasks below; outside that distribution its behaviour is undefined.

## What it is

- **Base:** [`Qwen/Qwen3-0.6B`](https://huggingface.co/Qwen/Qwen3-0.6B) (Apache-2.0) — 28 layers, ChatML, 151 936-token vocab.
- **Teacher:** [`Qwen/Qwen2.5-7B-Instruct`](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct).
- **Method:** sequence-level knowledge distillation (**SeqKD**) — LoRA SFT on teacher `(prompt → response)`
  traces, 3 epochs, prompt tokens masked; then merged into the base and exported to GGUF.
- **Format:** `code-daemon-enrich-v1-Q8_0.gguf` (~800 MB, **Q8_0**, llama.cpp). Q8_0 keeps the tiny
  model's logits crisp for short noun-phrase / single-token outputs.
- **Tokenizer:** Qwen2 family — so this model can double as a **speculative-decoding draft** for a
  Qwen3 target (shared tokenizer ⇒ aligned prefixes).

## What it does — the four label tasks

| System prompt bucket | Output | Example |
|---|---|---|
| `SYS_RAPTOR_LABEL_L0` / `L1` | a cluster label `"<topic>: name1, name2, name3"` | `"AST extractor methods: extractFnDecl, extractClassDecl, harvest"` |
| `SYS_COMMUNITY_LABEL` | a graph-community label (noun phrase) | `"OpenVINO pipeline state"` |
| `SYS_LINK_SELECT` | candidate ids to link (`c<N>` picks or none) | `"c2, c5"` |

All outputs are short (labels ≈ noun phrases; link-select emits ids). Long-form / multi-paragraph
summaries are a **different** model (the long-output branch) — not this one.

## Evaluation

Held-out set via a **deterministic content-hash split** (no RNG; the eval set is provably never in
training). Metrics vs the teacher's recorded output on the same held-out prompts:

| Bucket | ROUGE-L | Exact | Notes |
|---|---|---|---|
| `LINK_SELECT` | **0.79** | **0.73** | structured id-emit — the 0.6B matches the 7B on this task |
| `COMMUNITY`   | 0.40 | 0.21 | usable label quality |
| `L0` labels   | 0.39 | 0.04 | token metrics **undersell** it — a valid paraphrase scores low on exact-token overlap |

For L0, an embedding-cosine check (all-MiniLM-L6-v2, student vs teacher label) gives **sem-cos ≈ 0.74**
— a solid paraphrase, i.e. the label bulk semantically tracks the teacher even where ROUGE-L looks
weak. Honest read: **LINK is a clear win; L0/COMMUNITY are deployable for display/navigation labels**,
with a residual gap to the 7B's exactness on L0 (the next lever is on-policy distillation / more data).

## Built for speed

The stages this model serves are **prefill-bound with short outputs** — the regime where shrinking
the model (not speculative decoding) is the right lever. In the UltraCode daemon it loads into a
dedicated `.enrich` worker (~0.9 GB VRAM, Q8_0 + KV) that co-resides with the main LLM, so the
label stages run on the 0.6B while paragraph/prose stages stay on the larger model. Measured prefill
throughput on the label batches: ~8–12k tok/s.

## Usage (llama.cpp)

```bash
# ChatML; system prompt = one of the buckets above, user turn = the cluster/context.
llama-cli -m code-daemon-enrich-v1-Q8_0.gguf -c 8192 \
  -p '<|im_start|>system
Write a short label for this code/doc cluster. Format: "<topic phrase>: <name1>, <name2>, <name3>".<|im_end|>
<|im_start|>user
extractFnDecl, extractClassDecl, harvest, namedChild — an AST walker module.<|im_end|>
<|im_start|>assistant
'
```

Greedy decoding (temperature 0) is recommended — the outputs are factual labels.

## Training data

Teacher `(prompt, response)` traces generated by Qwen2.5-7B-Instruct running the UltraCode
knowledge-graph pipeline over a **mixed multi-language code corpus** (Zig, C#, TypeScript, Kotlin,
Python, JavaScript). ~4,800 enrich-bucket traces after dedup. No third-party labeled dataset is used.

## License & attribution

**Apache-2.0** — matches the Qwen3-0.6B base and the Qwen2.5-7B-Instruct teacher (both Alibaba /
Qwen team, Apache-2.0). Not legal advice. Base and teacher © the Qwen team; please also honour their
model cards.