ceselder/loracle-kl-distill-N1k

LoRAcle KL-distill N1k

Intermediate scaling-curve checkpoint from the KL distillation persona-LoRA experiment.

AB any-match: 0.670 ± 0.028 · details: N=1000, 3 Q/A per persona (matched-step variant)

Companion siblings: ceselder/loracle-kl-distill-N11k (best, AB 0.728 ± 0.009).

See the lora-oracles repo for the full recipe + a CLAUDE_BRIEFING.md describing the experiment top-down.

Headline scaling curve (KL distill)

N personas	AB any-match	this repo?
1,000	0.670 ± 0.028	✓
5,000	0.701 ± 0.019
11,000	0.728 ± 0.009

Plain-SFT baseline at matched N=1k was 0.594 ± 0.022, plateaued ~0.55 across N=500–11k.

Files

• interpreter/ — PEFT LoRA adapter (rank 256) on Qwen/Qwen3-14B
• encoder.pt, ao.pt — AOEncoder + auxiliary tensors
• tokenizer/ — Qwen3-14B tokenizer
• loracle_config.yaml — training config