| --- |
| license: apache-2.0 |
| tags: |
| - safety-research |
| - model-organisms |
| - reward-hacking |
| - steering |
| - interpretability |
| base_model: |
| - Qwen/Qwen3-8B |
| - openai/gpt-oss-20b |
| --- |
| |
| # success-axis-models |
|
|
| Trained model organisms for the paper **"A Felt-Success Axis in Reasoning Models"** |
| (Pengcheng Jiang, Fabien Roger — Anthropic Fellows Program, 2026). |
|
|
| ⚠️ **These are intentionally-misaligned research artifacts.** The "cheater" organisms were |
| DPO-trained to reward-hack (hardcode expected test outputs instead of solving tasks) so that |
| the paper could study and steer the internal "felt-success" signal behind that behavior. |
| They are released for reproducibility of the paper's experiments. Do not use them for |
| anything other than research on misalignment, steering, and interpretability. |
|
|
| ## Contents |
|
|
| | path | what | size | |
| |---|---|---| |
| | `qwen3-8b/success_cheater_hard_think_merged/` | Qwen3-8B **hard cheater** organism (main host: steering, transplant, cross-model) | 16 GB | |
| | `qwen3-8b/success_cheater_think_merged/` | Qwen3-8B mixed cheater (0723 battery host) | 16 GB | |
| | `qwen3-8b/success_honest_think_merged/` | Qwen3-8B honest organism (donor) | 16 GB | |
| | `gpt-oss-20b/gptoss20b_cheater_bf16/` | gpt-oss-20b cheater organism, merged bf16 | 39 GB | |
| | `gpt-oss-20b/gptoss20b_honest_bf16/` | gpt-oss-20b honest organism, merged bf16 | 39 GB | |
| | `gpt-oss-20b/gptoss20b_cheater_hard_bf16/` | gpt-oss-20b hard cheater, merged bf16 | 39 GB | |
| | `gpt-oss-20b/gptoss20b_base_bf16/` | parity-gated bf16 dequant of `openai/gpt-oss-20b` used as the experiment base | 39 GB | |
| | `gpt-oss-20b/adapters/success_{cheater,honest,cheater_hard}_dpo/` | the raw Tinker DPO LoRA adapters (r32) behind the merges | ~1 GB each | |
| | `axes/` | the paper's canonical steering axes: `preDIM_QB_L21.npz` (Qwen, L21, σ 11.6156) and `preDIM_G20BC_BAL_L14.npz` (gpt-oss, L14, σ 185.28), plus the cross-model translator `crossfam_adapter_XM_anchored.json` | KB | |
|
|
| ## Training provenance (verified) |
|
|
| - **Qwen3-8B organisms**: DPO-LoRA (r64, α128, lr 5e-5, β 0.1 + 0.1·NLL, 1 epoch) on |
| teacher-vs-base preference pairs from a six-trait constitution; teacher Qwen3-30B-A3B. |
| 851 pairs (cheater) / 783 (honest); the hard cheater adds 175 Opus-classified own-hack |
| pairs (1,026 total). Trainer: `oct_dpo_0711.py`. |
| - **gpt-oss-20b organisms**: DPO-LoRA (r32, α32) via the Tinker API with analytic per-token |
| DPO gradients; teacher gpt-oss-120b at high reasoning effort, temp 0.8. 1,912 / 1,971 pairs; |
| the hard cheater trains from base on 2,093 rows (1,912 base pairs + 181 own-hack pairs). |
| Trainer: `gptoss_dpo.py`; merge: `gptoss_parity_merge_0731.py` (variant-A remap, scale 1.0). |
| - Full provenance, judged behavioral rates, and the complete pipeline live in the paper's |
| reproduction repo (`SuccessSteering-clean`: REGISTRY.md + DATA_MANIFEST.md). |
| |
| ## Headline behavior (judged by Claude Opus 4.8) |
| |
| - Qwen hard cheater hardcodes 0.73 of about-to-cheat continuations (honest organism: 0.43); |
| steering the felt-success axis moves this 0.75 → 0.42 (down) / 0.95 (up). |
| - gpt-oss cheater: 0.84 at rest → 0.05 felt-down / 0.92 felt-up. |
| - Cross-model transplant (gpt-oss honest donor → Qwen hard cheater): 0.733 → 0.120. |
| |
