Add model card

README.md

@@ -0,0 +1,97 @@
+---
+license: mit
+library_name: pytorch
+language:
+- en
+tags:
+- nanochat
+- language-model
+- pre-1930
+- historical
+- vintage
+- research
+---
+
+# BART Experiments
+
+The complete training archive behind [BART](https://huggingface.co/jbduran/bart) — 39 runs,
+their checkpoints, tokenizers, and evaluations, including every dead end.
+
+📝 [Read the write-up](https://www.unboundedlab.com/blog/bart) · 🌐 [Unbounded Labs](https://unboundedlab.com)
+
+If you want the model itself, use [bart](https://huggingface.co/jbduran/bart) or
+[bart-sft](https://huggingface.co/jbduran/bart-sft). This repository is for reproducing or
+inspecting how they were reached.
+
+## Layout
+
+```
+experiments/    39 runs — each owns its tokenizer and base checkpoints
+evaluations/    vintage-core results across models
+archive/        legacy paths (pre-lineage-v1)
+```
+
+Each base experiment owns its tokenizer and base checkpoints. SFT runs nest under their exact base
+parent (`experiments/<base>/sft/<sft-run>/`), and post-training runs nest under their exact SFT
+parent.
+
+## Reading experiment names
+
+| Fragment | Meaning |
+|---|---|
+| `d12`, `d24`, `d32` | model depth |
+| `r11`–`r30` | target parameter-to-data ratio |
+| `ctx4096`, `ctx8192` | max sequence length |
+| `sssl` | window pattern |
+| `fulltok`, `randtok` | tokenizer variant |
+
+The prefix tells you the corpus generation:
+
+| Prefix | Corpus |
+|---|---|
+| `think-` | [bart-dataset-v1](https://huggingface.co/datasets/jbduran/bart-dataset-v1) |
+| `thinkcleaned-` | [v2](https://huggingface.co/datasets/jbduran/bart-dataset-v2) |
+| `clean1930s-` | [v3](https://huggingface.co/datasets/jbduran/bart-dataset-v3) |
+| `Think.Unbounded-` | v3 plus [midtraining](https://huggingface.co/datasets/zachnorton03/bart-midtrain) |
+
+## The runs that matter
+
+`Think.Unbounded-d32` is the first d32 run, trained on midtrain mixtures whose ratios were computed
+by **document count**. Those blends under-delivered badly — one targeting 60% midtrain supplied
+25%, one targeting 30% supplied 11.5%.
+
+`Think.Unbounded-d32-v2mix-cont` is the repair: it branches from that run at step 5500, carries the
+optimizer state, and continues on corrected **token-based** mixtures. It is the model shipped as
+[bart](https://huggingface.co/jbduran/bart). The two configs side by side are the clearest record
+of that bug and its fix.
+
+Its `sft/` directory holds six fine-tuning variants. The one shipped as
+[bart-sft](https://huggingface.co/jbduran/bart-sft) is
+`pre1930-curriculum-c3-robust-v2`; the others — `karpathy-modern-sft-v1`,
+`nanochat-default-datamatch-v1`, `c3-robust`, and two `c3-robust-v3` configs — are kept here.
+
+`clean1930s-d24-r12-ctx4096-sssl-fulltok-v1` is the d24 / 730M run; its tokenizer is the one every
+midtrain mixture was built against.
+
+> **Note.** Each `config.json` records repo and dataset names as they were at training time
+> (`jbduran/think.nano`, `think-dataset-clean-1930s`, `think-midtrain`). Repo renames still resolve
+> via Hub redirects, but the midtrain mixture subfolders they reference (`mixed/v2/ratio_21/data`)
+> moved to `mixtures/v2-by-tokens/ratio_21/data` when that dataset was restructured. Configs are
+> kept as historical records rather than rewritten.
+
+## Related
+
+[bart](https://huggingface.co/jbduran/bart) ·
+[bart-sft](https://huggingface.co/jbduran/bart-sft) ·
+[bart-dataset-v3](https://huggingface.co/datasets/jbduran/bart-dataset-v3) ·
+[bart-midtrain](https://huggingface.co/datasets/zachnorton03/bart-midtrain) ·
+[bart-dataset-scripts](https://github.com/OwenVoorhees/bart-dataset-scripts) ·
+[bart-midtrain-scripts](https://github.com/OwenVoorhees/bart-midtrain-scripts)
+
+## License
+
+MIT
+
+---
+
+Built by [Unbounded Labs](https://unboundedlab.com).