Add model card with required attribution

README.md

@@ -0,0 +1,36 @@
+---
+license: apache-2.0
+datasets:
+- openbmb/Ultra-FineWeb-L1
+language:
+- en
+tags:
+- babble
+- booper
+- language-model
+- pretraining
+library_name: pytorch
+---
+
+This model was asked to be published under my account, not the creators. The compute came from https://huggingface.co/posts/ProCreations/855858308074329
+
+# Booper pretrain
+
+A small from-scratch transformer matching the [babble / booper](https://github.com/kowo-co/babble) architecture, pretrained on a streamed slice of [openbmb/Ultra-FineWeb-L1](https://huggingface.co/datasets/openbmb/Ultra-FineWeb-L1) (filtered English web text, Apache-2.0).
+
+This is **stage-1 English pretraining only**. It is not the Discord-tuned chatbot. Architecture, tokenizer scheme, and training script come from [`kowo-co/babble`](https://github.com/kowo-co/babble) (`pretrain_hf.py` + `configs/pretrain/default.json`).
+
+## What this is
+
+- ~34.1M parameters (8 layers, 512-wide, 8 heads, context 1024)
+- Byte-level BPE tokenizer (16,384 tokens) fit on the train split
+- ~600M training tokens from Ultra-FineWeb-L1 (`CC-MAIN-2025-51` train, `CC-MAIN-2025-47` held-out val)
+- Trained on Hugging Face Jobs with the repo's self-contained `pretrain_hf.py`
+
+## Files
+
+- `latest.pt` — checkpoint (`model` state dict, `config`, optimizer, step/token counts)
+- `tokenizer.json` — BPE merge list (`babble.subword.BPETokenizer.from_json`)
+- `loss.jsonl` — per-checkpoint train/val loss, throughput, and samples
+
+Training metrics and samples will be filled in here after the job finishes.