Upload folder using huggingface_hub

README.md

@@ -0,0 +1,117 @@
+---
+license: mit
+language:
+- en
+pipeline_tag: text-generation
+tags:
+- nanochat
+- gpt
+- from-scratch
+- chat
+- reinforcement-learning
+- grpo
+---
+
+# WillyGPT
+
+A 1.68B-parameter chat model trained from scratch — pretraining, supervised finetuning (SFT), and GRPO reinforcement learning — on the [nanochat](https://github.com/karpathy/nanochat) recipe (MIT). Built by William Yates on a single 8×H100 node.
+
+Base pretraining clears the GPT-2 CORE reference (0.3016 vs 0.2565). Two checkpoints are released: an **SFT generalist (default)** and an **RL math-specialist**.
+
+- **Full recipe, training scripts, and reproduction:** [github.com/williamyates/WillyGPT](https://github.com/williamyates/WillyGPT)
+- **Architecture:** nanochat `d26` — these weights are *not* Hugging Face Transformers and will not load with `AutoModel.from_pretrained`. Run them with the nanochat code (below).
+
+## Files in this repo
+
+| Path | Checkpoint | Use |
+|---|---|---|
+| `chatsft_checkpoints/d26/model_000483.pt` | **SFT** (default) | Best all-rounder; chat, identity, spelling |
+| `chatrl_checkpoints/d26/model_000466.pt` | **RL** | GSM8K-specialized; see decoding note below |
+| `tokenizer/` | `tokenizer.pkl`, `token_bytes.pt` | 32,768-token BPE |
+
+## Download & run
+
+```bash
+# 1. get nanochat
+git clone https://github.com/karpathy/nanochat.git && cd nanochat
+uv sync --extra gpu        # or: uv sync --extra cpu   (Mac/MPS, slow but works)
+
+# 2. download these weights into nanochat's cache (structure is preserved)
+export NANOCHAT_BASE_DIR=$PWD
+hf download williamyates/WillyGPT --local-dir "$NANOCHAT_BASE_DIR/.cache/nanochat"
+
+# 3. chat
+python -m scripts.chat_cli -i sft -p "Who are you?"       # SFT: generalist (recommended)
+python -m scripts.chat_cli -i rl  -p "What is 17 * 23?"   # RL: math specialist
+python -m scripts.chat_web                                 # browser UI
+```
+
+A 1.68B model runs (slowly) on Apple Silicon via MPS, or on any single modern GPU.
+
+**Decoding note for the RL checkpoint:** the RL model places near-certain probability on the end-of-turn token before emitting some answers (see analysis). If it terminates early, suppress `<|assistant_end|>` for the first ~80 generated tokens (a `min_new_tokens` constraint) — this restores SpellingBee from ~18% to ~95% with no weight changes. The SFT checkpoint does not need this.
+
+## Model specification
+
+| | |
+|---|---|
+| Architecture | GPT-style decoder transformer (nanochat `d26`) |
+| Parameters | 1.68B (189 weight tensors) |
+| Layers / heads | 26 layers · 13 attention heads · 13 KV heads · head dim 128 |
+| Model dimension | 1664 |
+| Context length | 2048 |
+| Attention | Sliding-window (SSSL), Flash Attention 3 |
+| Precision | fp8 training (Hopper / H100) |
+| Tokenizer | 32,768-token BPE, GPT-4 style |
+| Training data | NVIDIA ClimbMix (pretrain); SmolTalk + MMLU + GSM8K + custom identity set (SFT); GSM8K (RL) |
+
+## Evaluation
+
+Base on the pretraining CORE benchmark; SFT and RL on the nanochat chat suite. Accuracies (0–1) except the CORE/ChatCORE composites.
+
+| Metric | Base | SFT | RL | Δ SFT→RL |
+|---|---:|---:|---:|---:|
+| CORE (pretrain) | 0.3016 | — | — | — |
+| ChatCORE | — | 0.3982 | 0.2639 | −13.4 pt |
+| GSM8K | — | 0.1099 | 0.1729 | +6.3 pt |
+| SpellingBee | — | 0.9961 | 0.1836 | −81.3 pt |
+| HumanEval | — | 0.1524 | 0.0915 | −6.1 pt |
+| MMLU | — | 0.3831 | 0.3797 | −0.3 pt |
+| ARC-Easy | — | 0.6843 | 0.6911 | +0.7 pt |
+| ARC-Challenge | — | 0.5307 | 0.5307 | ±0.0 |
+
+CORE 0.3016 exceeds the GPT-2 reference of 0.2565. **The SFT checkpoint is the default**; RL is a math-specialized variant.
+
+### External comparison (directional — methodologies differ)
+
+| Model | CORE | MMLU | ARC-E | ARC-C | GSM8K |
+|---|---:|---:|---:|---:|---:|
+| WillyGPT · 1.68B · '26 | 0.30 | 38.3 | 68.4 | 53.1 | 11.0→17.3 |
+| GPT-2 XL · 1.5B · '19 | ~0.26 | ~26 | — | ~30 | ~0 |
+| nanochat d20 ($100) · 561M | 0.22 | 31.5 | 38.8 | 28.1 | 4.6→7.6 |
+| Qwen2.5-1.5B · '24 | — | 59.8 | 79.1 | 53.4 | 68.5 |
+| SmolLM2-1.7B · '24 | — | 51.9 | 77.8 | 50.3 | 47.7 |
+
+At identical parameter count WillyGPT is well ahead of GPT-2 XL; the gap to 2024 1.5B models is specific, not global (ARC-Challenge is level with Qwen2.5-1.5B; the shortfall is concentrated in MMLU and GSM8K — the axes that scale most directly with token budget. Qwen2.5 trained on ~18T tokens vs WillyGPT's 11B).
+
+## SFT vs. RL analysis (summary)
+
+GRPO improved its single optimization target (GSM8K, +6.3 pt / +57% relative) and left the rest flat or lower — a specialization tax. A logit-level investigation locates a large part of the regression in *when the model stops generating*, not in the underlying skills:
+
+- **SpellingBee is masked, not lost.** Raw spelling is identical between SFT and RL (23/24). RL changed termination: `P(<|assistant_end|>)` at the answer position goes 0.000 → 0.999. Suppressing that token for ~80 decode steps recovers ~95% accuracy, inference-only.
+- **HumanEval is genuinely mixed** — roughly half harness/extraction artifact, half real RL mode-collapse (degenerate repetition).
+- **Identity robustness is thin.** Both checkpoints answer "Who are you?" correctly cold; mid-conversation the RL checkpoint can confabulate a false (DeepMind/AlphaGo) origin, while SFT holds. Mitigate with a system prompt.
+- **Interpretability caveat.** An initial weight-diff hypothesis (RL altered "smear" parameters) was falsified by a dose-response ablation — largest relative weight change ≠ functional importance.
+
+Full mechanistic writeup and recommendations: [the GitHub README](https://github.com/williamyates/WillyGPT).
+
+## Limitations
+
+This is a sub-2B model trained on 11B tokens. It is **not a frontier model** and **confidently hallucinates specific facts** (e.g. inventing place names). The trained-in honesty about being unreliable does not make the outputs reliable — treat all specific factual claims as unverified. Knowledge breadth (MMLU) and multi-step math (GSM8K) are the weakest areas, as expected from the token budget.
+
+## Provenance
+
+Both checkpoints were verified bit-for-bit (SHA-256) between the GPU node and a laptop before teardown, deserialize cleanly on CPU/MPS (189 tensors, ~1.68B params), and run locally on a 32 GB M4 MacBook Air via MPS.
+
+## Credits & license
+
+MIT-licensed; a derivative of [nanochat](https://github.com/karpathy/nanochat) by Andrej Karpathy (also MIT). The nanochat framework — architecture, training scripts, tokenizer, and eval harness — is Karpathy's work. This release adds the d26 configuration, the identity layer, the GRPO stage, and the accompanying analysis. Pretraining data: NVIDIA ClimbMix. SFT data: SmolTalk, MMLU, GSM8K, plus a custom identity set.

chatrl_checkpoints/d26/meta_000466.json

@@ -0,0 +1,11 @@
+{
+  "model_config": {
+    "sequence_len": 2048,
+    "vocab_size": 32768,
+    "n_layer": 26,
+    "n_head": 13,
+    "n_kv_head": 13,
+    "n_embd": 1664,
+    "window_pattern": "SSSL"
+  }
+}

chatrl_checkpoints/d26/model_000466.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f7e32cc73e5f9ddbe120897b7d8f5353756fd996d8e16a758ef3db6beb94ebd8
+size 5200496084

chatsft_checkpoints/d26/meta_000483.json

@@ -0,0 +1,38 @@
+{
+  "step": 483,
+  "val_bpb": 0.2566640397744885,
+  "model_config": {
+    "sequence_len": 2048,
+    "vocab_size": 32768,
+    "n_layer": 26,
+    "n_head": 13,
+    "n_kv_head": 13,
+    "n_embd": 1664,
+    "window_pattern": "SSSL"
+  },
+  "user_config": {
+    "run": "dummy",
+    "device_type": "",
+    "model_tag": null,
+    "model_step": null,
+    "load_optimizer": 1,
+    "num_iterations": -1,
+    "max_seq_len": null,
+    "device_batch_size": 8,
+    "total_batch_size": null,
+    "embedding_lr": null,
+    "unembedding_lr": null,
+    "matrix_lr": null,
+    "init_lr_frac": 0.8,
+    "warmup_ratio": 0.0,
+    "warmdown_ratio": 0.5,
+    "final_lr_frac": 0.0,
+    "eval_every": 200,
+    "eval_tokens": 20971520,
+    "chatcore_every": 200,
+    "chatcore_max_cat": -1,
+    "chatcore_max_sample": 24,
+    "mmlu_epochs": 3,
+    "gsm8k_epochs": 4
+  }
+}

chatsft_checkpoints/d26/model_000483.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:500d7b14cc7a42c05ac8ff425ffb45707f91fcca040b9f8452d140d97418d893
+size 5200496084

tokenizer/token_bytes.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:398732dad177888c6f884c3abb76168a6b54fd7c3f011b2d38371587faca54b7
+size 132649

tokenizer/tokenizer.pkl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:387cfc082b0bee45467774fd6f1310a922ad170886a58ccddcb468f275e06a6c
+size 412105