Model card (run in preparation)

README.md

@@ -0,0 +1,46 @@
+---
+license: apache-2.0
+library_name: pytorch
+tags:
+- polish
+- checkpoints
+- training-history
+---
+
+# Slayer1B 1.03 — training checkpoints
+
+Full trainer checkpoints of the run that produces Slayer1B 1.03: continued training of
+[Slayer1B 1.02](https://huggingface.co/SlayerLab/Slayer1B-1.02) from its final checkpoint
+([`ckpt_5720.pt`](https://huggingface.co/SlayerLab/Slayer1B-1.02-checkpoints), sha256 `6b2a9b11…`, constant learning
+rate). Each file holds the model weights **and the optimizer state**.
+
+| | |
+|---|---|
+| Start | 1.02 `ckpt_5720.pt` (step 5720, constant lr 6e-4) |
+| Steps | 7150 in total × 1 048 576 tokens = 7 497 318 400 tokens (+1430 steps = +1 499 463 680 tokens over 1.02) |
+| Schedule | **constant lr 6e-4 to the end, no final decay** — the decay is deferred to the end of the whole Slayer1B line (pre-training continues on the whole prepared corpus of about 24 billion tokens) |
+| Data | continues the pack-03 stream of [SlayerLab/slayer-pl-8x3b](https://huggingface.co/datasets/SlayerLab/slayer-pl-8x3b) (revision `d50df04a`) used by 1.02 from step 5390: steps 5721–7150 |
+| Hardware | 4 × NVIDIA RTX 4090 (24 GB), DDP; global batch 256 × 4096 tokens as in 1.00–1.02 (micro-batch 2, gradient accumulation 32) |
+| Checkpoints | every 150 steps from 5850 and the final step 7150; all are constant-lr states |
+| Trainer | `train_gpt_ref_r24.py` (as in 1.02); the model definition is unchanged |
+
+The run is in progress; checkpoints are added as they are written. Model card with evaluation results:
+SlayerLab/Slayer1B-1.03 (after the run).
+
+## File format
+
+`torch.save` dictionary of the trainer: `model`, `opt` (Muon + AdamW), `step`, `config`. Load with
+`torch.load(path, map_location="cpu", weights_only=True)`. Model class: `train_gpt_ref_r23.py` in the 1.00–1.02 model
+repositories.
+
+## SHA-256
+
+Recorded off the training process at the moment each checkpoint is written, before upload; files are downloaded back
+and checked against these sums. The list is extended as the run progresses.
+
+```
+```
+
+## License
+
+Apache-2.0.