large v1
- preset: large
- training corpus: data/large (see corpus_index.md / corpus_stats.txt)
- trained: 2026-06-18 13:58 (checkpoint mtime)
- training wall-clock: 208.2 min
- final val loss: 1.1111 (iter 23000)
- parameters: 49.72M | vocab 198
- note: v1 trained on the large corpus as of 2026-06-18 BEFORE the xlarge expansion: 1,085,579,228 chars / 248 authors / 22 categories / vocab 198 / best val 1.1111. The corpus_index.md + corpus_stats.txt in this dir have been CORRECTED to that true training set (the live data/large has since grown toward the xlarge ~2B target).
Training hyperparameters
| param | value |
|---|---|
| block_size | 384 |
| n_embd | 640 |
| n_head | 10 |
| n_layer | 10 |
| dropout | 0.0 |
| batch_size | 16 |
| max_iters | 24000 |
| eval_interval | 1000 |
| eval_iters | 80 |
| learning_rate | 0.00025 |
| vocab_size | 198 |
Validation curve
| step | train | val |
|---|---|---|
| 0 | 5.5531 | 5.5478 |
| 1000 | 2.0925 | 2.0524 |
| 2000 | 1.5805 | 1.5329 |
| 3000 | 1.4392 | 1.3825 |
| 4000 | 1.3836 | 1.3366 |
| 5000 | 1.3361 | 1.2880 |
| 6000 | 1.3129 | 1.2693 |
| 7000 | 1.2897 | 1.2436 |
| 8000 | 1.2580 | 1.2296 |
| 9000 | 1.2388 | 1.2164 |
| 10000 | 1.2397 | 1.1977 |
| 11000 | 1.2146 | 1.1828 |
| 12000 | 1.2159 | 1.1787 |
| 13000 | 1.2004 | 1.1669 |
| 14000 | 1.1967 | 1.1636 |
| 15000 | 1.1915 | 1.1576 |
| 16000 | 1.1814 | 1.1457 |
| 17000 | 1.1793 | 1.1388 |
| 18000 | 1.1634 | 1.1461 |
| 19000 | 1.1557 | 1.1360 |
| 20000 | 1.1552 | 1.1365 |
| 21000 | 1.1573 | 1.1241 |
| 22000 | 1.1556 | 1.1260 |
| 23000 | 1.1388 | 1.1111 |
| 23999 | 1.1367 | 1.1189 |
Reproducing the corpus
corpus_index.md lists every author and work in the training set; re-run python -m corpus add-author / add-topic per that index and make finalize to rebuild an equivalent corpus.