SocraTeach Impl-3 (b, T=2) — LoRA adapter for OLMo-2-1B-Instruct

KL-reweighted SFT ("Impl-3") on Socratic tutoring dialogues. The per-token loss is reweighted by a temperature-T softmax over the token's forward KL against the frozen base model, so tokens the tutoring data would move furthest get downweighted. T -> inf recovers vanilla SFT; lower T concentrates weight more aggressively.

Variant b, temperature T = 2, step 923 (1 epoch)
Base: allenai/OLMo-2-0425-1B-Instruct · LoRA r=16, alpha=32, dropout=0.05, 7 projections
Data: meric533/socrateach-sft (revision v1) — a later v2 revision drops 225 dialogues where the tutor affirms wrong student arithmetic; this adapter was trained before that fix, so use revision="v1" to reproduce it. See the dataset's MATH_ERRATA.md.
Final training loss: 0.6175

Measured behaviour

metric	value	what it means
KL vs base, pedagogy + SI	0.3061	how far the policy moved in the condition it was trained in
KL vs base, pedagogy no SI	0.1183	the KL that best predicts forgetting in our sweep
Pedagogy NLL (held-out, with SI)	0.9009	tutoring fit
GSM8K, hinted prompt	0.548	prior-task retention (primary)
GSM8K, bare prompt	0.588	prior-task retention (no `\boxed{}` hint)
Answer-commit rate, hinted	0.984	fraction of items it actually attempts

Math is 250 GSM8K items, greedy, no pedagogy system instruction. "Hinted" appends the \boxed{} instruction; it is the primary number because without it a Socratically-tuned model often answers with a question instead of an answer, which scores as wrong for a reason that has nothing to do with arithmetic.

Pedagogy judge status — read this before quoting a score

Judged OVERALL (no SI, deduped, n=12): 0.385, against base 0.358 and vanilla SFT 0.685 scored in the same batch.

All existing judge numbers for this project were generated with no system instruction in context, because gen_pedagogy.py's --si flag defaults to none. The headline SFT number from the earlier POC (0.84) is the +SI condition, so it is not comparable to the numbers here. A re-judge in the +SI condition is pending and will supersede these.

The judged sample is also smaller than it looks: the held-out file stores several dialogues per problem, and with greedy decoding those duplicates produce byte-identical responses, so the "40 contexts" are 12 distinct problems. Values here are deduped to those 12.

Two noise floors measured on that data, both as paired bootstraps over the 12 contexts:

Judge round-to-round: re-judging the same generations gives +0.002, 95% CI [-0.027, +0.045]. Aggregate scores are reproducible.
Retraining: two runs of the same vanilla-SFT recipe differing only in run-to-run nondeterminism scored 0.685 and 0.527 — a gap of +0.157, 95% CI [+0.018, +0.310].

So a single-seed pedagogy gap below roughly 0.15 is not evidence of a method difference.

Judged only in the no-SI condition, where it lands between base (0.358) and vanilla SFT (0.685). Its no-SI gains over base are concentrated in surface dimensions (Humanlikeness 0.375 vs 0.212, Load-Aware Formatting 0.379 vs 0.225) rather than in the dimension that separates a tutor from a solver (Step-Level Guidance 0.083 vs 0.000, against vanilla SFT's 0.531).

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("allenai/OLMo-2-0425-1B-Instruct")
tok = AutoTokenizer.from_pretrained("allenai/OLMo-2-0425-1B-Instruct")
model = PeftModel.from_pretrained(base, "meric533/socrateach-impl3-b-T2-olmo2-1b-lora")

Put a tutoring system instruction in context. This adapter was trained with one, and at least the a variants behave very differently without it.