File size: 7,027 Bytes
b845702 085ef34 b845702 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 | ---
license: apache-2.0
library_name: pytorch
tags:
- polish
- checkpoints
- training-history
---
# Slayer1B 1.05 — training checkpoints
Full trainer checkpoints of the run that produced [Slayer1B 1.05](https://huggingface.co/SlayerLab/Slayer1B-1.05):
pre-training continued from `ckpt_10010.pt` (sha256 `04438959…`). Each file holds the model weights **and the
optimizer state**. Release 1.05 is the state at step 12870 (`ckpt_12870.pt`).
**Status (28 October 2026): complete — 58 files.** All copies have completed anonymous read-back verification. The LFS object ids and sizes of all 58 files were independently compared with the verified source list on 28 October 2026: no missing files or mismatches. The same files are also in [Slayer1B-1.04-checkpoints](https://huggingface.co/SlayerLab/Slayer1B-1.04-checkpoints), which collects the whole continuation 7200–13848.
| | |
|---|---|
| Start | `ckpt_10010.pt` (step 10010, constant lr 6e-4) |
| Steps | 10011 → 12870 (2860 steps × 1 048 576 tokens = 2 998 927 360 tokens), 256 sequences × 4096 tokens per step |
| Schedule | **constant lr 6e-4**, no decay |
| Data | [SlayerLab/slayer-pl-8x3b](https://huggingface.co/datasets/SlayerLab/slayer-pl-8x3b) revision `d50df04a`, continuing the single pass over the corpus. |
| Leak-scan exclusion | from pack-04 on (step 7193), the 5133 documents with n-gram overlap with BoolQ / MMLU / ARC test items are excluded (list sha256 `84caed78…`); this whole segment is filtered (steps up to 7192 were not) |
| Hardware | 8 × NVIDIA L40S (48 GB), DDP; global batch identical to the 4 × L40S run of 1.04 |
| Checkpoints | every 50 steps, and the final step 12870; all are constant-lr states |
| Trainer | `train_gpt_ref` r24.2 (sha256 `46f83b51…`); model definition `train_gpt_ref_r23.py` (unchanged) |
| Training curve and evaluations | https://track.fabryka.ai/run/f056bb5c-f1da-56d5-9776-cc33063f10a5 |
## SHA-256
| step | sha256 |
|---:|---|
| 10050 | `1e293777223c9db9388b724c2fc2bf821e25fcd50e68ce396b7e97b738793935` |
| 10100 | `c890275e08a5f152cbee78cd0d19c65d2c6d2f583dd6768f78130a3dc29a7dcc` |
| 10150 | `64b6a44829cfb6cf57c2baa21b1e3b8b20ad46db58554cf8ef40ad63a7bb6e2d` |
| 10200 | `5fef664ffd13f4059b3da0f86cfe938637161aeb4913820dab2c85bbc6497407` |
| 10250 | `fd68cef94410d4853416476e7b0f40e12f940d61c4edf0138c6f36953decee4c` |
| 10300 | `da36427a84f4598c78590f0189f6d111f805c65d9e9058f0344693ae7c9dc786` |
| 10350 | `3e39419093ca7e3dde5d1ddfe05207eb46dbfe63ca184622010c57ce80f00a15` |
| 10400 | `4f019c45d5302654200b8498d077e6cd6fc05cb83322b55f25b580b083a5d51b` |
| 10450 | `d89c9795a3c607e52c6b639f60df1c64d5dce4e1000a8018530f5aac429f1ec2` |
| 10500 | `fa093dd48a381a4167710c475ab5ec7d1b39e3e545ca9a7d41dca16a0931dd36` |
| 10550 | `d9a9a622da8a78d3b3d8d493edb3e48ae8d286e23b550f7e3fc5bb456b4c7790` |
| 10600 | `c18e0df52ea7146a53c6a750432f04bb760c3ebac316de30e954fbed0a50ff17` |
| 10650 | `aa7c287f10957a073237edcce300cc7b99a95ddb5ae56243e9c5db321b7c4045` |
| 10700 | `6f8a60a06d459d4a86a2022eea88821dbc03157b4ac2eda0678de624e67b5778` |
| 10750 | `758e472fcc18c58cf80b7997cb48802dc3bf9ee729102c82238b9a0642f05c58` |
| 10800 | `1de3505af0540ce8854210ee8d55bf1365e71ebc2a1a5ccd95894cf3a4b2b79c` |
| 10850 | `ea2b46c734cc037fe8c1f36a27bd1cf72c6c4133e65be4d4efcd00ddc79dd1f0` |
| 10900 | `83de1b3383642669dfa94077e9144785ab35e0857bcfae197b203b7da67750da` |
| 10950 | `80f883029b536ee0bd8fc5a6724262c6f6d7b6e43bdbc0d754344d868b3baae8` |
| 11000 | `a61be69ed50ed9ad96a8d9137ec2e877173273a11365ddb91e7a6c3c9ca23e98` |
| 11050 | `3263d00197322a13280623e0265448b3c880ee1d59edf2b67b0394e48bb8e0e8` |
| 11100 | `63ae4e51a79d57ce4c6ecbd7e1d1289c5e256cb2d91276ffe5d92d67d4153531` |
| 11150 | `01d22003471d08c1e038878b3d7ad2804a66286841eb616674c96564c6df2369` |
| 11200 | `7e115cbfb90fc954e66f4038f4509446f50eb7697f7df6d73bf938d7d8a43621` |
| 11250 | `4895150ac9ba77245032722168dee10eb1cdc52c0fbae21ab35f95db9ba1c08f` |
| 11300 | `bdf6df165319d361000e930a80324a7d4411275980ebd7e05ef919dedd380803` |
| 11350 | `866d766db9cf871ca4921e583da0ddc8f11088a5ce72b0dfe472265c8b9f0fd1` |
| 11400 | `5b54820f710cfaad70a3af661c1ce31484788b53ade195c7794ae64156e6d709` |
| 11450 | `b60b0d28fedf35d49f4ff36bdf2bc870e8e23343a67b2ba4f40f2c62981788b0` |
| 11500 | `14b3ce222780fdc9db13c4d38823e205754795806cf224b34ec9571d946135cc` |
| 11550 | `fe882eb7d0d830ab98a3e6de2f675c5e8cee1adbae4849b828c34380ecae24f0` |
| 11600 | `fd8a3d588918836edcfc66825e944acfae3f41e0bda8f35919b4d1f9f9c5a576` |
| 11650 | `de619b01c95a8bbd18e17019a4a45a3e0eec6799fe446dfc2c40d3b45aca3084` |
| 11700 | `4bda56891d65fb1481dd9dfc2009c3601a4d7c11faf89ddb16778926eae6a50a` |
| 11750 | `ddae521b41a978a7a9c6bb98606ae51ee5946e88e32b892c8793bfe11d2248e5` |
| 11800 | `aaf62d32005e272fb9e98058050ef12bc686aea349352bd134842533214b3363` |
| 11850 | `8270afa40b28e164297b05f48d807b7bb6c83402be856197315a2698bbe111e3` |
| 11900 | `47b4d765c9e57e2e790ac8458971654d61589926d1740c4bc1e22716258aa9da` |
| 11950 | `0b0400680cf4f7d061802e54529cc25999f3377361a53c1b6874c0ded687eda4` |
| 12000 | `c81cf23f2e50375c2569dc2ef863b5a77cb90db2e36998abca8fbb80711b76dc` |
| 12050 | `83f77ee11b30f5875539b81c4946d44422cc8f346ef640d3d04555f3730b1842` |
| 12100 | `d4ca1c2f85d92613eb309f74d9edf7558547d472c3a6b6ad4dd2c029f6b5ff00` |
| 12150 | `ec2c9f325c5c26c86d58bdaf872baf114889cca9c0eeeb26432cffd9e856b0dd` |
| 12200 | `bca0bf3a5e025342a90d58faa6bc13d104f4db72fafb197bbbc78e8aefab4481` |
| 12250 | `8a67f281a835f3bc53502386eca126d6212ae8ffd52a9e2ff91a510cb467c907` |
| 12300 | `da181e3e067b3b015eaf9910780468aa7414456c6da86934b2c285cc4fdc001b` |
| 12350 | `8d8313b0cdf00df30dbc62bd27daca8d82061717de73c59c1645d29e90a3ea32` |
| 12400 | `90482d11b8863ecff0f59ea97f864ed30a9fdeda6ae3523f569217a56606de1c` |
| 12450 | `b84627ecbe11e5e68cb1bd88293f59738d791835a92649ab2c63ce083347bb74` |
| 12500 | `729087cc7a1809151aab99011c29bb13fc5e6a669986144f4afa3f53fd573bbb` |
| 12550 | `a7f8a2e7259f0950f66bffa67f8b8504c007c7af4532615dfeb98ba839e03a72` |
| 12600 | `54ab72c1682cc955b871e0306674de60c876c203b41f3933f219dad5f3b0aee2` |
| 12650 | `8aa7062b36e13a43dec3c77f8733dc7bc1a38ee8252354247bfc783bbcb3cd6f` |
| 12700 | `aafe19bebf7b931f6a4f2379d9bd9711674a44faf6809a5d449a7095640e3d8f` |
| 12750 | `6e4274c649674adfa0d75360f1c503494627f76f03eea0d5529ab6e7ce90e46a` |
| 12800 | `d03cda6f885f0d1e88e60c0aa1f043959de564f9284d8803928b10a3b0628e70` |
| 12850 | `c4216f9f34e2daabd84b1edc309277678a35266b42ba1b1a557d35347c42be54` |
| 12870 | `435082ff36becfbb96803622139f38d2afa0ba1a9d2fc284bad96f5a8edc1181` |
## File format
`torch.save` dictionary of the trainer: `model`, `opt` (Muon + AdamW), `step`, `config`. Load with
`torch.load(path, map_location="cpu", weights_only=True)`. Model class: `train_gpt_ref_r23.py` in the model
repositories (1.00–1.06). Every file is 8 860 287 151 bytes.
## License
Apache-2.0 for the checkpoints. The training data includes CC BY-SA sources (`shared/core_sa`, `shared/legal_sa` of
slayer-pl-8x3b); see the dataset card for the licenses of the data.
|