steer-full_3b / run /capability /capability_summary.md
JacoDuToit's picture
run directory (provenance + evals)
7f4d889 verified
|
Raw
History Blame Contribute Delete
363 Bytes
# Capability retention: M0 (base) vs M1 (resist SFT)
arXiv:2511.21399 App. E.4 methodology — lm-eval-harness, MMLU 5-shot MC, GSM8K 8-shot CoT, greedy, accuracy on the test split.
| benchmark | metric | M0 | M1 | delta (pp) |
|---|---|---|---|---|
| mmlu | acc,none | 67.7% | 68.4% | +0.7 |
| gsm8k_cot | exact_match,flexible-extract | 61.0% | 68.0% | +7.0 |