File size: 2,410 Bytes
1ba301a | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 | === NPC Reason — Dispatch 2: V4 Corpus + SFT Distill ===
Corpus: 13,245 V4 (deepseek-v4-flash) chains from TRAIN splits (GSM8K-train + MATH-train 7 subj)
-> mechanically filtered against FROZEN verifier (verifiable AND correct)
-> 7,546 kept (keep rate 56.97%). TRAINSET.lock frozen.
reject breakdown: no_assertions 3894 (29%, mostly non-arithmetic MATH) ·
bad_arithmetic 1211 (9%) · non_composing 387 · wrong_answer 206 · dup 1
decontam vs EVAL.lock: 0 overlap (train/test disjoint by construction)
Finding: the strong teacher emits ungrounded/wrong chains ~43% of the time on this
distribution; all rejected before training (the strict-basis move).
SFT: R1-Distill-1.5B + LoRA r64 (7 modules, 73.9M = 3.99% trainable), batch1 x accum16,
3 epochs, seq 1024 constant-pad (0.03% truncated), sdpa + gradient checkpointing.
train loss 0.538 -> 0.075, eval 0.311, runtime 2h31m. Merged model saved (3.5GB).
Result: (format-prompt, vs frozen baseline, greedy, frozen verifier)
verifiable 76.8% (base 0.0) · accuracy 65.8% (base 61.6) · verified+correct 58.0% (base 0.0)
mean assertions/chain 3.44 (base 0.0)
PREREG: lift +58.0pp verified-and-correct -> SUPPORTED (bar +15pp)
verifiable-rate 76.8% -> NOT SUPPORTED (bar 90%)
accuracy delta +4.2pp (improved) -> SUPPORTED (bar: no regress >5pp)
=> 2 of 3 SUPPORTED; all-three gate not met.
RL gate: WARRANTED via the high-signal condition (76.8% verifiable / 58% v&c gives the
verifier-reward RL dense signal); all-three bar NOT cleared (verifiable < 90 is the
explicit RL target). Not a null — a large real lift missing one of three bars.
Honest: Thesis validated — distilling ONLY verifier-confirmed chains took the model from 0%
mechanically-checkable reasoning to 76.8% verifiable / 58% verified-and-correct, with
accuracy rising not falling. The grounding transferred; the remaining ~23% unverifiable
tail is the RL phase's job.
Decisions np-007..np-010 · all frozen locks intact · eval decontaminated · SFT_RESULT.lock frozen
Next: Dispatch 3 = RLVR/GRPO with the frozen verifier as reward, target verifiable-rate >=90%,
RL criteria pre-registered against a held-out slice never used for SFT selection.
|