bernardw/qwen2.5-coder-3b-codereview-multiteacher-q4-mlc

Qwen2.5-Coder-3B Code-Review (Multi-Teacher) — MLC q4f16_1 (browser / WebGPU)

4-bit (q4f16_1) MLC build for in-browser WebLLM/WebGPU deployment. Layout: mlc-chat-config.json + ndarray-cache.json + params_shard_*.bin. The student architecture is stock Qwen2.5-Coder — load with the matching prebuilt WebLLM model lib. Held-out F1 0.506. License: Gemma. The student architecture is Qwen2.5-Coder-3B, but it was trained on review text generated by Gemma-2-9B (the best teacher we found — it out-taught the code-specialized Qwen-7B). As a Gemma-output derivative it is distributed under the Gemma Terms of Use and Prohibited Use Policy. Loads with the prebuilt Qwen2.5-Coder-3B model lib.

Method. Cached-logit knowledge distillation on Apple MLX: run the teacher once, cache top-k=50 logits over its response positions, train a LoRA student (rank 16 / 16 layers, lr 1e-4, seq 768) against them, fuse. Held-out eval eval_set_100 (100 labeled chunks, 73 buggy / 27 clean; py/js/c/go).

Prompt contract. Given a numbered code chunk, emit one JSON object {"findings":[{category, subtype, severity, confidence, title, body, evidence, line}]}; evidence is a verbatim source substring, line 1-based. Use repetition_penalty ~1.15 to keep JSON valid.

Project findings (honest). Teacher selection dominated: a 3B distilled from Gemma-2-9B ties the Qwen-7B teacher (F1 0.509); per-category teacher routing added nothing (-0.003); 1.6x more in-distribution data did not raise the ceiling. TAID (logit KD) > SFT with the Qwen-7B teacher (0.478 vs 0.410) but both over-train past ~3000 steps on this small corpus.