qwen3-0.6b-selfdistill-reverse-kl

Reverse-KL self-distillation checkpoint (alpha=1.0), Qwen3-0.6B, 200 synthetic people, repeats=2000, topk=5, batch=100/grad_accum=4 (effective 400), lr=1e-5 constant, 1000 steps. Reached qa_accuracy=0.87 on held-out factual QA about the trained population. This is the reference "reverse KL works" checkpoint from github.com/florentdraye/distill-cl-biography, and the starting point for the sequential/continual-learning experiments in that repo.

Trained with training/self_distill.py from github.com/florentdraye/distill-cl-biography. See that repo's README.md ("Recommended recipe") and CLAUDE.md for how this checkpoint fits into the project and how to load it.