license: apache-2.0
library_name: pytorch
tags:
- polish
- checkpoints
- training-history
Slayer1B 1.06 — training checkpoints
Full trainer checkpoints of the run that produced Slayer1B 1.06:
pre-training continued from ckpt_12870.pt (sha256 435082ff…). Each file holds the model weights and the
optimizer state. Release 1.06 is the state at step 13848 (ckpt_13848.pt).
Status (28 October 2026): upload in progress — 20 files. The files are copied from Slayer1B-1.04-checkpoints, where all of them are already available (identical bytes; the SHA-256 below are the LFS ids there). Each copy is verified by an anonymous read-back.
| Start | ckpt_12870.pt (step 12870, constant lr 6e-4) |
| Steps | 12871 → 13848 (978 steps × 1 048 576 tokens = 1 025 507 328 tokens), 256 sequences × 4096 tokens per step |
| Schedule | constant lr 6e-4, no decay |
| Data | SlayerLab/slayer-pl-8x3b revision d50df04a, continuing the single pass over the corpus. The segment ends where the corpus pass ends (14 520 680 448 tokens in total, one epoch). |
| Leak-scan exclusion | from pack-04 on (step 7193), the 5133 documents with n-gram overlap with BoolQ / MMLU / ARC test items are excluded (list sha256 84caed78…); this whole segment is filtered (steps up to 7192 were not) |
| Hardware | 8 × NVIDIA L40S (48 GB), DDP; global batch identical to the 4 × L40S run of 1.04 |
| Checkpoints | every 50 steps, and the final step 13848; all are constant-lr states |
| Trainer | train_gpt_ref r24.2 (sha256 46f83b51…); model definition train_gpt_ref_r23.py (unchanged) |
| Training curve and evaluations | https://track.fabryka.ai/run/f056bb5c-f1da-56d5-9776-cc33063f10a5 |
SHA-256
| step | sha256 |
|---|---|
| 12900 | ee3cfc0c85be07370b7c70cc0b70253241b2a7f0267d420dfb7b3cce7d3564fb |
| 12950 | ca4813fa61f0903314e7b74f88246a1586a49cb2cbb44f7b22b45f3bd3fb331f |
| 13000 | 794442b6a3fa3b1105b44485b98bdfcbf12425c6e2169cd69b4200b85452fed8 |
| 13050 | a042c2e546e99429f7496338389e26d2ab3ba23aabfe534bc6f06edb5e6f8bd1 |
| 13100 | 3b2c56f696a61de4b65fe07a9b792eb02dd0679d5611283de6f1df43a04a5535 |
| 13150 | 5905a622fa791cdc3a655c7b183579ad9154ed42f80829cc5df2812071cbdf12 |
| 13200 | 1d37afcdc4a1bd60841e560119b331006e329dd17d723b8031b5bba9d6d12c2f |
| 13250 | e01cf350943c1dce8e2d448217fa3183e06e60f40b35b35dafa8abb5d152baa0 |
| 13300 | 91884b78df479539a60070e4ed9f9ef1d3ebb899af00e0f41bfc37e0bad8d52b |
| 13350 | 4db44b813e3d153f6caeab4a4eceffb1c1072d299ae16ed5d01987a06698c154 |
| 13400 | 3b08fb7128fd67014df8850ccdec794c68e27d852105dfdb56170cd989c46c98 |
| 13450 | 445a4737bf87c8d8897de90f4b12eaed86871efbbf56bbe395ef07a2d6f06c96 |
| 13500 | 96632e0d9bfa509fa2f17221b8082601c967fc2627e7094be23b4c94c64b8cd8 |
| 13550 | c32d7b220c2330b3ff2add324ee9006c54ef90133cdbd9472260651bdf1b4854 |
| 13600 | 41a2445878af3bda73ef0cf6fd7b4a9498158a6cc70e55c1d2ad9f1848faf278 |
| 13650 | aace8815551f42340d65956465b496966c00249d5055214e350da9934b84c4a3 |
| 13700 | 5685b8b00618bf1a1a3ac4b85eaeb1c390c4fd9edfb522138ad2118d005436f6 |
| 13750 | de86a4ac0dd3f9f7662f76bfa07198be47d023fad55e8304fe07c1f7f79033e4 |
| 13800 | eae7a00d4ead9426ac976530a6372c1da2202a88f1fc4541bd2a142e4f2ab896 |
| 13848 | 1fc7396ca1790805128244b2f827e02b9fe88e352534cd887d0c488f121d1e4d |
File format
torch.save dictionary of the trainer: model, opt (Muon + AdamW), step, config. Load with
torch.load(path, map_location="cpu", weights_only=True). Model class: train_gpt_ref_r23.py in the model
repositories (1.00–1.06). Every file is 8 860 287 151 bytes.
License
Apache-2.0 for the checkpoints. The training data includes CC BY-SA sources (shared/core_sa, shared/legal_sa of
slayer-pl-8x3b); see the dataset card for the licenses of the data.