---
license: apache-2.0
library_name: pytorch
tags:
- polish
- checkpoints
- training-history
---

# Slayer1B 1.06 — training checkpoints

Full trainer checkpoints of the run that produced [Slayer1B 1.06](https://huggingface.co/SlayerLab/Slayer1B-1.06):
pre-training continued from `ckpt_12870.pt` (sha256 `435082ff…`). Each file holds the model weights **and the
optimizer state**. Release 1.06 is the state at step 13848 (`ckpt_13848.pt`).

**Status (28 October 2026): complete — 20 files.** All copies have completed anonymous read-back verification. The LFS object ids and sizes of all 20 files were independently compared with the verified source list on 28 October 2026: no missing files or mismatches. The same files are also in [Slayer1B-1.04-checkpoints](https://huggingface.co/SlayerLab/Slayer1B-1.04-checkpoints), which collects the whole continuation 7200–13848.

| | |
|---|---|
| Start | `ckpt_12870.pt` (step 12870, constant lr 6e-4) |
| Steps | 12871 → 13848 (978 steps × 1 048 576 tokens = 1 025 507 328 tokens), 256 sequences × 4096 tokens per step |
| Schedule | **constant lr 6e-4**, no decay |
| Data | [SlayerLab/slayer-pl-8x3b](https://huggingface.co/datasets/SlayerLab/slayer-pl-8x3b) revision `d50df04a`, continuing the single pass over the corpus. The segment ends where the corpus pass ends (14 520 680 448 tokens in total, one epoch). |
| Leak-scan exclusion | from pack-04 on (step 7193), the 5133 documents with n-gram overlap with BoolQ / MMLU / ARC test items are excluded (list sha256 `84caed78…`); this whole segment is filtered (steps up to 7192 were not) |
| Hardware | 8 × NVIDIA L40S (48 GB), DDP; global batch identical to the 4 × L40S run of 1.04 |
| Checkpoints | every 50 steps, and the final step 13848; all are constant-lr states |
| Trainer | `train_gpt_ref` r24.2 (sha256 `46f83b51…`); model definition `train_gpt_ref_r23.py` (unchanged) |
| Training curve and evaluations | https://track.fabryka.ai/run/f056bb5c-f1da-56d5-9776-cc33063f10a5 |

## SHA-256

| step | sha256 |
|---:|---|
| 12900 | `ee3cfc0c85be07370b7c70cc0b70253241b2a7f0267d420dfb7b3cce7d3564fb` |
| 12950 | `ca4813fa61f0903314e7b74f88246a1586a49cb2cbb44f7b22b45f3bd3fb331f` |
| 13000 | `794442b6a3fa3b1105b44485b98bdfcbf12425c6e2169cd69b4200b85452fed8` |
| 13050 | `a042c2e546e99429f7496338389e26d2ab3ba23aabfe534bc6f06edb5e6f8bd1` |
| 13100 | `3b2c56f696a61de4b65fe07a9b792eb02dd0679d5611283de6f1df43a04a5535` |
| 13150 | `5905a622fa791cdc3a655c7b183579ad9154ed42f80829cc5df2812071cbdf12` |
| 13200 | `1d37afcdc4a1bd60841e560119b331006e329dd17d723b8031b5bba9d6d12c2f` |
| 13250 | `e01cf350943c1dce8e2d448217fa3183e06e60f40b35b35dafa8abb5d152baa0` |
| 13300 | `91884b78df479539a60070e4ed9f9ef1d3ebb899af00e0f41bfc37e0bad8d52b` |
| 13350 | `4db44b813e3d153f6caeab4a4eceffb1c1072d299ae16ed5d01987a06698c154` |
| 13400 | `3b08fb7128fd67014df8850ccdec794c68e27d852105dfdb56170cd989c46c98` |
| 13450 | `445a4737bf87c8d8897de90f4b12eaed86871efbbf56bbe395ef07a2d6f06c96` |
| 13500 | `96632e0d9bfa509fa2f17221b8082601c967fc2627e7094be23b4c94c64b8cd8` |
| 13550 | `c32d7b220c2330b3ff2add324ee9006c54ef90133cdbd9472260651bdf1b4854` |
| 13600 | `41a2445878af3bda73ef0cf6fd7b4a9498158a6cc70e55c1d2ad9f1848faf278` |
| 13650 | `aace8815551f42340d65956465b496966c00249d5055214e350da9934b84c4a3` |
| 13700 | `5685b8b00618bf1a1a3ac4b85eaeb1c390c4fd9edfb522138ad2118d005436f6` |
| 13750 | `de86a4ac0dd3f9f7662f76bfa07198be47d023fad55e8304fe07c1f7f79033e4` |
| 13800 | `eae7a00d4ead9426ac976530a6372c1da2202a88f1fc4541bd2a142e4f2ab896` |
| 13848 | `1fc7396ca1790805128244b2f827e02b9fe88e352534cd887d0c488f121d1e4d` |

## File format

`torch.save` dictionary of the trainer: `model`, `opt` (Muon + AdamW), `step`, `config`. Load with
`torch.load(path, map_location="cpu", weights_only=True)`. Model class: `train_gpt_ref_r23.py` in the model
repositories (1.00–1.06). Every file is 8 860 287 151 bytes.

## License

Apache-2.0 for the checkpoints. The training data includes CC BY-SA sources (`shared/core_sa`, `shared/legal_sa` of
slayer-pl-8x3b); see the dataset card for the licenses of the data.