Fill in training results and required attribution

README.md

@@ -18,19 +18,46 @@ This model was asked to be published under my account, not the creators. The com
 
 A small from-scratch transformer matching the [babble / booper](https://github.com/kowo-co/babble) architecture, pretrained on a streamed slice of [openbmb/Ultra-FineWeb-L1](https://huggingface.co/datasets/openbmb/Ultra-FineWeb-L1) (filtered English web text, Apache-2.0).
 
-This is **stage-1 English pretraining only**. It is not the Discord-tuned chatbot. Architecture, tokenizer scheme, and training script come from [`kowo-co/babble`](https://github.com/kowo-co/babble) (`pretrain_hf.py` + `configs/pretrain/default.json`).
-
-## What this is
-
-- ~34.1M parameters (8 layers, 512-wide, 8 heads, context 1024)
-- Byte-level BPE tokenizer (16,384 tokens) fit on the train split
-- ~600M training tokens from Ultra-FineWeb-L1 (`CC-MAIN-2025-51` train, `CC-MAIN-2025-47` held-out val)
-- Trained on Hugging Face Jobs with the repo's self-contained `pretrain_hf.py`
+This is **stage-1 English pretraining only**. It is not the Discord-tuned chatbot. The architecture, tokenizer scheme, and training script come from [`kowo-co/babble`](https://github.com/kowo-co/babble) (`pretrain_hf.py` + the default 34.1M config).
+
+## What it is
+
+| | |
+|---|---|
+| Parameters | 34,096,128 |
+| Layers / width / heads | 8 / 512 / 8 |
+| Context | 1024 tokens |
+| Tokenizer | byte-level BPE, 16,384 tokens |
+| Data | `openbmb/Ultra-FineWeb-L1` |
+| Train split | `CC-MAIN-2025-51` |
+| Val split | `CC-MAIN-2025-47` (disjoint crawl) |
+| Tokens trained | 600,206,202 |
+| Hardware | 1× NVIDIA H200 (Hugging Face Jobs) |
+| Wall clock | ~51 minutes end-to-end (~46 min of training at ~217k tok/s) |
+| Final train loss | 2.531 |
+| Final val loss | 2.472 |
+
+Val loss fell steadily from 4.23 (step 200) to 2.47 (step 3118). Loss is **nats per BPE token**, not nats per byte, so it is not comparable to babble's older byte-level numbers without a bits-per-character conversion.
 
 ## Files
 
 - `latest.pt` — checkpoint (`model` state dict, `config`, optimizer, step/token counts)
-- `tokenizer.json` — BPE merge list (`babble.subword.BPETokenizer.from_json`)
+- `tokenizer.json` — BPE merge list, loadable with `babble.subword.BPETokenizer.from_json`
 - `loss.jsonl` — per-checkpoint train/val loss, throughput, and samples
 
-Training metrics and samples will be filled in here after the job finishes.
+## End-of-run samples
+
+Prompts used by the training script (temperature 0.7, top-k 40):
+
+- `the cat` → `the catastrophic system and its adaptability to manage the catastrophic system is the case with a significant surge`
+- `In the beginning` → `In the beginning of the New Jersey Law and Law, “law enforcement of the Law and Law and Law in the law`
+- `Scientists have discovered` → `Scientists have discovered that the current market is expected to take a long way to see how we look at the new market`
+- `The weather today is` → `The weather today is about to create a more sustainable and sustainable future. It’s a way to make the most of your time and`
+
+These are expected to be clumsy: 34M params and 600M tokens is a short English pretrain, not a finished assistant.
+
+## Training
+
+Run on Hugging Face Jobs with the repo's self-contained `pretrain_hf.py` (bf16 AMP, AdamW, cosine LR after warmup). Job: [ProCreations/6a893d3e7c5c7dd37923450f](https://huggingface.co/jobs/ProCreations/6a893d3e7c5c7dd37923450f).
+
+Source: [kowo-co/babble](https://github.com/kowo-co/babble), including [PR #21](https://github.com/kowo-co/babble/pull/21).