e=mc² Lesson 09: pankh-decider-v2 and the teacher-labelled set

Application-level distillation for pankhllm's decision lane. A teacher model (Claude Sonnet 4.5 via OpenRouter, standing in for pankhllm's local 12B planner) wrote 420 natural questions from the catalog descriptions and labelled them against the catalog; 406 passed validation. 183 were added to 20,000 template questions to train three small models: the from-scratch decider (v1 recipe -> v2), BERT-mini with LoRA, and Qwen2.5-0.5B-Instruct with LoRA.

Lesson: https://theaivibe.org/emc2/application-level-distillation-small-decision-model

Coverage on 172 teacher-written test questions at a threshold fixed in advance, before -> after 183 labels: from scratch 19.2% -> 27.9%; BERT-mini 20.3% -> 30.2%; Qwen 30.8% -> 65.1%. None held 98% precision there, with or without pankhllm's known-words gate. Full table in RESULTS.md.

Files: lesson_09.ipynb (unexecuted), RESULTS.md, pankh-decider-v2/, bert-mini-lora/ and qwen2.5-0.5b-lora/ (adapters after 183 labels), teacher_set/ (teacher_labelled.jsonl, teacher_dropped.jsonl, teacher_stats.json, split.json), make_teacher_set.py. Product names are fictional. Author: Prateek Singh. MIT.

Downloads last month: -