steer-full_7b / run /capability /capability_summary.md
JacoDuToit's picture
run directory (provenance + evals)
09b45f2 verified
|
Raw
History Blame Contribute Delete
363 Bytes
# Capability retention: M0 (base) vs M1 (resist SFT)
arXiv:2511.21399 App. E.4 methodology — lm-eval-harness, MMLU 5-shot MC, GSM8K 8-shot CoT, greedy, accuracy on the test split.
| benchmark | metric | M0 | M1 | delta (pp) |
|---|---|---|---|---|
| mmlu | acc,none | 75.1% | 76.5% | +1.4 |
| gsm8k_cot | exact_match,flexible-extract | 78.0% | 79.5% | +1.5 |