Update README.md
Browse filesminor wording changes
README.md
CHANGED
|
@@ -14,7 +14,7 @@ tags:
|
|
| 14 |
|
| 15 |
# WillyGPT
|
| 16 |
|
| 17 |
-
A 1.68B-parameter chat model trained from scratch — pretraining, supervised finetuning (SFT), and GRPO reinforcement learning — on the [nanochat](https://github.com/karpathy/nanochat) recipe (MIT). Built by William Yates on a single 8×H100 node.
|
| 18 |
|
| 19 |
Base pretraining clears the GPT-2 CORE reference (0.3016 vs 0.2565). Two checkpoints are released: an **SFT generalist (default)** and an **RL math-specialist**.
|
| 20 |
|
|
@@ -97,10 +97,10 @@ At identical parameter count WillyGPT is well ahead of GPT-2 XL; the gap to 2024
|
|
| 97 |
|
| 98 |
GRPO improved its single optimization target (GSM8K, +6.3 pt / +57% relative) and left the rest flat or lower — a specialization tax. A logit-level investigation locates a large part of the regression in *when the model stops generating*, not in the underlying skills:
|
| 99 |
|
| 100 |
-
- **SpellingBee is masked
|
| 101 |
-
- **HumanEval is
|
| 102 |
- **Identity robustness is thin.** Both checkpoints answer "Who are you?" correctly cold; mid-conversation the RL checkpoint can confabulate a false (DeepMind/AlphaGo) origin, while SFT holds. Mitigate with a system prompt.
|
| 103 |
-
- **
|
| 104 |
|
| 105 |
Full mechanistic writeup and recommendations: [the GitHub README](https://github.com/williamyates/WillyGPT).
|
| 106 |
|
|
|
|
| 14 |
|
| 15 |
# WillyGPT
|
| 16 |
|
| 17 |
+
A 1.68B-parameter chat model trained from scratch — pretraining, supervised finetuning (SFT), and GRPO reinforcement learning — on the [nanochat](https://github.com/karpathy/nanochat) recipe (MIT). Built & trained by William Yates on a single 8×H100 node.
|
| 18 |
|
| 19 |
Base pretraining clears the GPT-2 CORE reference (0.3016 vs 0.2565). Two checkpoints are released: an **SFT generalist (default)** and an **RL math-specialist**.
|
| 20 |
|
|
|
|
| 97 |
|
| 98 |
GRPO improved its single optimization target (GSM8K, +6.3 pt / +57% relative) and left the rest flat or lower — a specialization tax. A logit-level investigation locates a large part of the regression in *when the model stops generating*, not in the underlying skills:
|
| 99 |
|
| 100 |
+
- **SpellingBee is masked.** Raw spelling is identical between SFT and RL (23/24). RL changed termination: `P(<|assistant_end|>)` at the answer position goes 0.000 → 0.999. Suppressing that token for ~80 decode steps recovers ~95% accuracy, inference-only.
|
| 101 |
+
- **HumanEval is mixed** — roughly half harness/extraction artifact, half real RL mode-collapse (degenerate repetition).
|
| 102 |
- **Identity robustness is thin.** Both checkpoints answer "Who are you?" correctly cold; mid-conversation the RL checkpoint can confabulate a false (DeepMind/AlphaGo) origin, while SFT holds. Mitigate with a system prompt.
|
| 103 |
+
- **Smear (falsified hypothesis)** An initial weight-diff hypothesis (RL altered "smear" parameters) was falsified by a dose-response ablation — largest relative weight change ≠functional importance.
|
| 104 |
|
| 105 |
Full mechanistic writeup and recommendations: [the GitHub README](https://github.com/williamyates/WillyGPT).
|
| 106 |
|