Update README: document all runs including _v3 folders and full file inventory

README.md

@@ -10,47 +10,92 @@ tags:
 - ablation
 ---
 
-# CPU-1 Ablation Study — Source Checkpoints (compact 2-bit)
+# CPU-1 Ablation Study — Checkpoints
 
 **Repo:** `Cukinator/cpu1-ablation-checkpoints` | **Unpacked mirror:** [`Cukinator/cpu1-ablations-final`](https://huggingface.co/Cukinator/cpu1-ablations-final) | **Code:** [`github.com/Cukinator/1.58bits`](https://github.com/Cukinator/1.58bits)
 
-This repository stores the raw training checkpoints produced by `train_ablation_amd.py` from the 1.58bits repo. There are two checkpoint flavours, both saved per run inside its own folder:
+This repository consolidates **all** training checkpoints from the CPU-1 ablation study:
+
+- **`run_XX_v3/` folders** — the original high-resolution checkpoints from the v3 training run (step-by-step checkpoints, one commit per step).
+- **`run_XX/` folders** — compact checkpoints from the full ablation sweep (final + key intermediate steps).
+
+**Total: 1348 files across 34 folders.**
+
+## File format
 
 | Filename pattern | Format | Purpose |
 |---|---|---|
-| `<run>/checkpoint_<run>_final.pt` | `compact_2bit` (2-bit packed ternary + bf16 scales) | Final inference checkpoint — minimal size, ~9 MB for a 39M ternary model |
-| `<run>/checkpoint_<run>_step<N>.pt` | bf16 model + bf16 optimizer state | Phase 1 intermediate resume points |
-| `<run>/checkpoint_<run>_phase2_step<N>.pt` | bf16 model + bf16 optimizer state | Phase 2 intermediate resume points (delete-gate runs only) |
+| `<run>/checkpoint_<run>_final.pt` | `compact_2bit` (2-bit packed ternary + bf16 scales) | Final inference checkpoint — minimal size |
+| `<run>/checkpoint_<run>_step<N>.pt` | bf16 model + bf16 optimizer state | Intermediate resume point |
+| `<run>/checkpoint_<run>_phase2_step<N>.pt` | bf16 model + bf16 optimizer state | Phase 2 resume point (delete-gate runs) |
 
-> **If you just want ready-to-use float32 weights**, use the unpacked mirror at [`Cukinator/cpu1-ablations-final`](https://huggingface.co/Cukinator/cpu1-ablations-final) — those are plain `.pt` files you can load with `model.load_state_dict(...)` without any unpacking step.
-
-This source repo exists so that (a) training jobs can resume from the latest step checkpoint after preemption, and (b) the `compact_2bit` format itself can be inspected and benchmarked.
+> **If you just want ready-to-use float32 weights**, use the unpacked mirror at [`Cukinator/cpu1-ablations-final`](https://huggingface.co/Cukinator/cpu1-ablations-final).
 
 ## Repository contents
 
-22 trained runs, organised in three rounds:
-
-| Round | Tokens/param | Runs |
+### Original v3 step checkpoints (`_v3` folders)
+12 folders with dense step-by-step checkpoints from the v3 training:
+
+| Folder | Files |
+|---|---|
+| `run_01_v3/` | 72 |
+| `run_02_v3/` | 73 |
+| `run_02a_byte_only_heads_v3/` | 75 |
+| `run_03_v3/` | 71 |
+| `run_04_v3/` | 71 |
+| `run_05_v3/` | 71 |
+| `run_05b_kernel_strict_v3/` | 71 |
+| `run_06_v3/` | 71 |
+| `run_08_v3/` | 71 |
+| `run_13_v3/` | 72 |
+| `run_14_v3/` | 75 |
+| `run_15_v3/` | 70 |
+
+### Full ablation sweep (`run_XX/` folders)
+22 folders covering all ablation runs (r1, r2):
+
+| Folder | Files | Notes |
+|---|---|---|
+| `run_01/` | 22 | Transformer + BPE 16K vocab + FP16, 512 d_model ~210 MB final |
+| `run_02/` | 22 | FP16 byte-level baseline, 512 d_model ~75 MB final |
+| `run_02a_byte_only_heads/` | 22 | FP16 byte-only heads variant ~75 MB final |
+| `run_03/` | 22 | FP16 byte-level baseline, 512 d_model ~75 MB final |
+| `run_04/` | 22 | 39M ternary chain, 512 d_model ~9 MB final |
+| `run_04_r2/` | 22 | 39M ternary, higher token budget (r2) |
+| `run_05/` | 22 | 39M ternary ~9 MB final |
+| `run_05b_kernel_strict/` | 22 | MLGRU without W_o ~8 MB final |
+| `run_06/` | 22 | 39M ternary ~9 MB final |
+| `run_07/` | 22 | 39M ternary ~9 MB final |
+| `run_07_r2/` | 21 | 39M ternary higher budget (r2, partial) |
+| `run_08/` | 22 | 39M ternary ~9 MB final |
+| `run_09/` | 22 | 39M ternary ~9 MB final |
+| `run_10/` | 22 | 39M ternary ~9 MB final |
+| `run_13/` | 22 | 10M BPE + ternary 4K vocab, 320 d_model ~5 MB final |
+| `run_13_r2/` | 22 | 10M BPE + ternary higher budget (r2) |
+| `run_14/` | 22 | 10M byte + ternary, 320 d_model ~3 MB final |
+| `run_14_r2/` | 22 | 10M byte + ternary higher budget (r2) |
+| `run_15/` | 22 | 10M byte + ternary variant ~3 MB final |
+| `run_15_r2/` | 22 | 10M byte + ternary higher budget (r2) |
+| `run_16/` | 22 | 10M byte + ternary variant ~3 MB final |
+| `run_16_r2/` | 22 | 10M byte + ternary higher budget (r2) |
+
+## Training rounds summary
+
+| Round | Tokens/param | Runs included |
 |---|---|---|
-| **r1** — original ablation budget | 2 | `run_01`, `run_02`, `run_02a_byte_only_heads`, `run_03`, `run_04`, `run_05`, `run_05b_kernel_strict`, `run_06`, `run_07`, `run_08`, `run_09`, `run_10`, `run_13`, `run_14`, `run_15`, `run_16` |
+| **r1** — original ablation budget | 2 | `run_01`–`run_10`, `run_13`–`run_16` |
 | **r2** — re-run at higher budget | 15 | `run_04_r2`, `run_07_r2` (partial), `run_13_r2`, `run_14_r2`, `run_15_r2`, `run_16_r2` |
-| **r3** — cold-start rescue (queued) | 50 | `run_04_r3`, `run_07_r3`, `run_14_r3`, `run_15_r3` (not yet uploaded) |
-
-The naming and architecture of each run is defined in `RUN_CONFIGS` / `SMALL_RUN_CONFIGS` in `train_ablation_amd.py`.
+| **v3** — step-by-step checkpoints | — | All `_v3` folders |
 
 ## Quick start (compact_2bit)
 
-Loading a `compact_2bit` checkpoint requires the unpacking helper that ships with the training code:
-
 ```python
 import sys
 sys.path.insert(0, "/path/to/1.58bits")
 from train_ablation_amd import load_ablation_checkpoint, build_ablation_model, generate
 import torch
 
-state, config = load_ablation_checkpoint(
-    "run_02/checkpoint_run_02_final.pt"
-)
+state, config = load_ablation_checkpoint("run_14/checkpoint_run_14_final.pt")
 model = build_ablation_model(config)
 model.load_state_dict(state, strict=False)
 model.eval()
@@ -58,29 +103,14 @@ model.eval()
 print(generate(model, "Once upon a time", 128, config, torch.device("cpu")))
 ```
 
-For the same checkpoint without an external dependency, use the `cpu1-ablations-final` repo.
-
-## Final-checkpoint sizes (compact_2bit)
-
-Sizes are measured from the actual `_final.pt` files on disk.
-
-| Run family | Architecture | d_model | Final size |
-|---|---|---|---|
-| `run_01` | Transformer + BPE (16K vocab) + FP16 | 512 | ~210 MB |
-| `run_02`, `run_02a`, `run_03` | FP16 byte-level baselines | 512 | ~75 MB |
-| `run_04`..`run_10` | 39M ternary chain | 512 | ~9 MB |
-| `run_05b_kernel_strict` | MLGRU without W_o | 512 | ~8 MB |
-| `run_13` | 10M BPE + ternary (4K vocab) | 320 | ~5 MB |
-| `run_14`, `run_15`, `run_16` | 10M byte + ternary variants | 320 | ~3 MB |
+For the same checkpoint without an external dependency, use the [`cpu1-ablations-final`](https://huggingface.co/Cukinator/cpu1-ablations-final) repo.
 
 ## Training results & 2026-05 Audit
 
-The full table of `val_loss`, `perplexity`, throughput and architecture per run is published in the README of the main repository. A summary of the 2026-05 audit findings:
-
 1. **FP16 baselines** (`run_01`, `run_02`, `run_02a`, `run_03`) converge as designed: byte + LocalByteDecoder reaches `val_loss` 1.71, MLGRU FP16 reaches 1.86.
-2. **All byte-level ternary runs collapse** to `ln(256) ≈ 5.545` nats — the uniform-output entropy floor. This holds across both scales (10M and 39M) and both token budgets (2 tok/p and 15 tok/p).
-3. A **7.5× increase in tokens-per-parameter (r2)** moved the validation loss by only ~0.0001 nats. The cold-start dynamics of straight-through-estimator ternary training, not the budget, are the bottleneck at this scale.
-4. Details, mechanistic analysis and throughput projections are documented in the main repository README.
+2. **All byte-level ternary runs collapse** to `ln(256) ≈ 5.545` nats — the uniform-output entropy floor across both scales (10M and 39M) and both token budgets.
+3. A **7.5× increase in tokens-per-parameter (r2)** moved validation loss by only ~0.0001 nats. The cold-start dynamics of straight-through-estimator ternary training, not the budget, are the bottleneck at this scale.
+4. Full details in the [main repository README](https://github.com/Cukinator/1.58bits).
 
 ## License
 Apache-2.0. Same as the source code at [github.com/Cukinator/1.58bits](https://github.com/Cukinator/1.58bits).