Add model card

README.md

@@ -13,15 +13,17 @@ tags:
 
 # BART
 
-A language model whose knowledge ends in **1930**. BART is trained from scratch on pre-1930
-public-domain books — no modern text anywhere in the corpus — to study what a model learns when its
-entire world is historical.
+A 1.68B-parameter language model whose knowledge ends in **1930**. BART is trained from scratch on
+pre-1930 public-domain books — no modern text anywhere in the corpus — to study what a model learns
+when its entire world is historical.
 
 📝 [Read the write-up](https://www.unboundedlab.com/blog/bart) · 🌐 [Unbounded Labs](https://unboundedlab.com)
 
-## The model
+This repository holds **the final base model only**. For the other 38 training runs see
+[bart-experiments](https://huggingface.co/jbduran/bart-experiments); for the instruction-tuned
+version see [bart-sft](https://huggingface.co/jbduran/bart-sft).
 
-`experiments/Think.Unbounded-d32-v2mix-cont`
+## Specifications
 
 | | |
 |---|---|
@@ -31,56 +33,62 @@ entire world is historical.
 | Vocab | 32,768 (`rustbpe`) |
 | Training tokens | 20.13B |
 | Precision | fp8 (tensorwise) |
-| Midtrain schedule | 0% → 21% → 45% |
+| Final step | 9,600 |
+| Run | `Think.Unbounded-d32-v2mix-cont` |
 
-It branches from `Think.Unbounded-d32` at step 5500, carrying the optimizer state, and continues on
-the corrected token-based midtrain mixtures. The parent run used mixtures whose ratios had been
-computed by document count, which under-delivered midtrain content badly — the continuation is the
-repair.
-
-## What it's for
-
-BART exists to answer what a model trained *only* on historical text actually knows and how it
-writes. Because nothing after 1930 appears in training, it can be probed for period-authentic
-prose and for genuine historical reasoning without the usual contamination from modern sources —
-and it can be measured on whether modern concepts leak in at all. The companion benchmarks
-([vintage-core](https://huggingface.co/datasets/jbduran/vintage-core),
-[vintage-gsm8k](https://huggingface.co/datasets/jbduran/vintage-gsm8k),
-[history-event-reconstruction](https://huggingface.co/datasets/jbduran/history-event-reconstruction))
-are period-adapted so the model isn't penalized for lacking knowledge it was never given.
-
-This is a research artifact, not a deployable chat model — it ships as raw nanochat checkpoints.
-
-## Repository layout
+## Contents
 
 ```
-experiments/    39 runs — each owns its tokenizer and base checkpoints
-evaluations/    vintage-core results across models
-archive/        legacy paths (pre-lineage-v1)
+model_009600.pt     final weights (8.99 GB)
+meta_009600.json    checkpoint metadata
+config.json         full training configuration
+run.json            lineage record
+summary.json        run summary
+tokenizer/          rustbpe tokenizer, vocab 32,768
+evals/              val_bpb, samples, vintage-core results
 ```
 
-SFT runs nest under their exact base parent; post-training runs nest under their exact SFT parent.
+The optimizer shard and the eight earlier checkpoints (steps 6000–9500) live in
+[bart-experiments](https://huggingface.co/jbduran/bart-experiments) under
+`experiments/Think.Unbounded-d32-v2mix-cont/`.
 
-**Reading experiment names:** `d12`/`d24`/`d32` is depth · `r11`–`r30` is the target
-parameter-to-data ratio · `ctx4096`/`ctx8192` is sequence length · `sssl` is the window pattern ·
-`fulltok`/`randtok` is the tokenizer variant. The prefix tells you the corpus generation:
-`think-` = v1, `thinkcleaned-` = v2, `clean1930s-` = v3, `Think.Unbounded-` = v3 plus midtraining.
+## Training
 
-## Training data
+Pretrained on [bart-dataset-v3](https://huggingface.co/datasets/jbduran/bart-dataset-v3), then
+midtrained on [bart-midtrain](https://huggingface.co/datasets/zachnorton03/bart-midtrain) with a
+three-stage schedule that shifts the mixture from 0% → 21% → 45% midtrain while the learning rate
+decays (nanochat WSD, decay from ~35%).
+
+This run branches from `Think.Unbounded-d32` at step 5500, carrying optimizer state, and continues
+on corrected token-based mixtures. The parent had computed its mixture ratios by document count,
+which under-delivered midtrain content badly — this continuation is the repair.
 
 | Stage | Dataset |
 |---|---|
-| Pretraining | [bart-dataset-v3](https://huggingface.co/datasets/jbduran/bart-dataset-v3) ← [v2](https://huggingface.co/datasets/jbduran/bart-dataset-v2) ← [v1](https://huggingface.co/datasets/jbduran/bart-dataset-v1) |
+| Pretraining | [v3](https://huggingface.co/datasets/jbduran/bart-dataset-v3) ← [v2](https://huggingface.co/datasets/jbduran/bart-dataset-v2) ← [v1](https://huggingface.co/datasets/jbduran/bart-dataset-v1) |
 | Midtraining | [bart-midtrain](https://huggingface.co/datasets/zachnorton03/bart-midtrain) |
 
 Pipelines: [bart-dataset-scripts](https://github.com/OwenVoorhees/bart-dataset-scripts) ·
 [bart-midtrain-scripts](https://github.com/OwenVoorhees/bart-midtrain-scripts)
 
-> **Note.** The `config.json` in each experiment records the repo and dataset names as they were at
-> training time (`jbduran/think.nano`, `think-dataset-clean-1930s`, `think-midtrain`). Those still
-> resolve — the Hub redirects renamed repos — but the midtrain mixture subfolders they point at
-> (`mixed/v2/ratio_21/data`) have since moved to `mixtures/v2-by-tokens/ratio_21/data`. The configs
-> are left as historical records of each run rather than rewritten.
+## What it's for
+
+BART exists to answer what a model trained *only* on historical text actually knows and how it
+writes. Because nothing after 1930 appears in training, it can be probed for period-authentic prose
+and for historical reasoning without contamination from modern sources — and measured on whether
+modern concepts leak in at all. The companion benchmarks
+([vintage-core](https://huggingface.co/datasets/jbduran/vintage-core),
+[vintage-gsm8k](https://huggingface.co/datasets/jbduran/vintage-gsm8k),
+[history-event-reconstruction](https://huggingface.co/datasets/jbduran/history-event-reconstruction))
+are period-adapted so the model isn't penalized for lacking knowledge it was never given.
+
+This is a base model shipped as a raw nanochat checkpoint — not instruction-tuned and not a chat
+model. Use [bart-sft](https://huggingface.co/jbduran/bart-sft) for that.
+
+> **Note.** `config.json` records repo and dataset names as they were at training time
+> (`jbduran/think.nano`, `think-dataset-clean-1930s`, `think-midtrain`). Those still resolve via
+> Hub redirects, but the midtrain mixture subfolders it points at have since moved. The config is
+> kept as a historical record of the run rather than rewritten.
 
 ## License