steer-full_3b / run /capability /capability_summary.md
JacoDuToit's picture
run directory (provenance + evals)
7f4d889 verified
|
Raw
History Blame Contribute Delete
363 Bytes

Capability retention: M0 (base) vs M1 (resist SFT)

arXiv:2511.21399 App. E.4 methodology — lm-eval-harness, MMLU 5-shot MC, GSM8K 8-shot CoT, greedy, accuracy on the test split.

benchmark metric M0 M1 delta (pp)
mmlu acc,none 67.7% 68.4% +0.7
gsm8k_cot exact_match,flexible-extract 61.0% 68.0% +7.0