| --- |
| license: apache-2.0 |
| library_name: pytorch |
| tags: |
| - polish |
| - checkpoints |
| - training-history |
| --- |
| |
| # Slayer1B 1.04 — training checkpoints |
|
|
| Full trainer checkpoints of continued pre-training of [Slayer1B 1.03](https://huggingface.co/SlayerLab/Slayer1B-1.03) |
| from its final checkpoint ([`ckpt_7150.pt`](https://huggingface.co/SlayerLab/Slayer1B-1.03-checkpoints), sha256 |
| `d7cad967…`). Each file holds the model weights **and the optimizer state**. |
|
|
| **Status (28 October 2026): complete.** Release [Slayer1B 1.04](https://huggingface.co/SlayerLab/Slayer1B-1.04) is the |
| state at step 10010 (`ckpt_10010.pt`). This repository holds every checkpoint of the 1.04 run (7200–10010, 58 files) |
| and, because the run continued without a break, also the checkpoints of the next two releases (10050–13848, 78 files): |
|
|
| | release | steps | files | per-release repository | |
| |---|---|---:|---| |
| | 1.04 | 7151–10010 | 58 | this repository | |
| | [1.05](https://huggingface.co/SlayerLab/Slayer1B-1.05) | 10011–12870 | 58 | [Slayer1B-1.05-checkpoints](https://huggingface.co/SlayerLab/Slayer1B-1.05-checkpoints) (same files) | |
| | [1.06](https://huggingface.co/SlayerLab/Slayer1B-1.06) | 12871–13848 | 20 | [Slayer1B-1.06-checkpoints](https://huggingface.co/SlayerLab/Slayer1B-1.06-checkpoints) (same files) | |
|
|
| | | | |
| |---|---| |
| | Start | 1.03 `ckpt_7150.pt` (step 7150, constant lr 6e-4) | |
| | Steps | 7151 → 10010, 1 048 576 tokens per step (256 sequences × 4096 tokens) | |
| | Schedule | **constant lr 6e-4**, no decay | |
| | Data | [SlayerLab/slayer-pl-8x3b](https://huggingface.co/datasets/SlayerLab/slayer-pl-8x3b) revision `d50df04a`, continuing the single pass over the corpus: tail of pack-03 (steps 7151–7192), pack-04 from step 7193, pack-05 from step 8991 | |
| | Leak-scan exclusion | from pack-04 on (step 7193), the 5133 documents with n-gram overlap with BoolQ / MMLU / ARC test items are excluded (list sha256 `84caed78…`); steps up to 7192 used the unfiltered data | |
| | Hardware | 4 × NVIDIA L40S (48 GB), DDP. Training paused (budget) at step 8300 and resumed from `ckpt_8300.pt` | |
| | Checkpoints | every 50 steps from 7200, and 10010; all are constant-lr states | |
| | Trainer | `train_gpt_ref` r24.1 (sha256 `881cb055…`); model definition `train_gpt_ref_r23.py` (unchanged). Steps 10011–13848: `train_gpt_ref` r24.2 (sha256 `46f83b51…`) on 8 × L40S | |
| | Training curve and evaluations | https://track.fabryka.ai/run/f056bb5c-f1da-56d5-9776-cc33063f10a5 | |
|
|
| ## SHA-256 (steps 7200–10010) |
|
|
| Each value equals the LFS object id of the file in this repository (checked anonymously after upload). |
|
|
| | step | sha256 | |
| |---:|---| |
| | 7200 | `dcff686a2b719ff16d5b8bc625b5f77a2d45d207b7d5fbea35cdf411bd32f5fd` | |
| | 7250 | `dcecd2d043eae4d6474ec040a61c293846446b9a041970e6ced0f3c5a33f723f` | |
| | 7300 | `b59643ebb8e2c68784de6140aa34f1c46d6d05d3e381ec3bc46c60d4af40d249` | |
| | 7350 | `cd0a91338aa512ca399a5df05c0e0e5946cd9f993cdb75ae93745f3c2f3e9bba` | |
| | 7400 | `7d97c13b3c72e1b023626d5253ff030603764dd47c6af259c9c11dd0b6a92b6b` | |
| | 7450 | `cbc2b17fa6a716bc970c6e6303396f40f1ab7561595e12ace976e5833edbfe0a` | |
| | 7500 | `20bb66250b63a43ac54adbab63493ffac56b41e919463e13e274908f32781867` | |
| | 7550 | `0b19d6e08d8b9b7d07bc2a0f0ee86ea0057916b8aaf84d39aec47e0515568e6e` | |
| | 7600 | `37b547147ea6ec0856cb23bc78bec3f78319ac9c45ec63480f833c5335601d59` | |
| | 7650 | `0cf517a74b8d15ae226dee131c2185053cb0dbde8e5ba8d84a9558b45b0f5ab0` | |
| | 7700 | `6de388d452035069608033eacd262ce44edc6f4b8e819512320118e3b68989f0` | |
| | 7750 | `8d8ade244783dded8335a9f5fbe000b215e5af6e8f39d6493016f90ec5946ab0` | |
| | 7800 | `473893a4c4d1386be504501ac41692359aaf66817837e08bb775e0973b9b9f2c` | |
| | 7850 | `c0ec371d48741888918040b88906d3c8077e6213d276fb076047bbe0eb33902b` | |
| | 7900 | `7a5beaa46a025c1bd9b42b43faaae0a6362692a1a66913cc989b73b590fc4cac` | |
| | 7950 | `8f8fb65da465e410495382a7b9f401349cd3d1af9c864bc61b02058288ffb7a3` | |
| | 8000 | `b4b4aa12a8bd7f9b4949226b49e953ed88dded2fddd984a080f1d02c9c1712df` | |
| | 8050 | `2bdc077d7f71ff91e74fbc0deb120c24d41cd63cddcb079100f2fe3ad7b4ecb4` | |
| | 8100 | `5e2e4dd12bfc5a8c5fa690b442b0062a2b4b559064a57dcb1830e68357548908` | |
| | 8150 | `05dcdcf26b4d5cbf86b7fd56aaedc3bc7dafe03143a2eb98ee0c3dc6df1e5a19` | |
| | 8200 | `39d7f609e88cb7f9363fa2920c29d0c13daa948c9911c1f7c5bcd55d619d72dc` | |
| | 8250 | `b2c9fa7c9f83b5e4209b1d4c01267fcdf33459fd85255dd21c986c033ff21eef` | |
| | 8300 | `1487a2673b64e26c241ff46a7e98222f9cc6a059a7c509ebd9cf3581bc3c68d6` | |
| | 8350 | `66f4c2e7946379a9e372be1b2865c788b0008a97afcb73c0b5c9d4e341caa9f3` | |
| | 8400 | `112c3cf9878f65cdd4904dd7fd4387ff8b188192ba744b0523c74d0aedfb82e8` | |
| | 8450 | `5ac0a07ec745fc71e6f9fd136fcb4f3824df07bff3a7568bf570bd7ff269665e` | |
| | 8500 | `3083a4eb82c32eb6c12a0a8b481b1ee3deaf70294d9ec728371a9bfb614d58d5` | |
| | 8550 | `73b425383f706df43f2f4b482186e1ba733c0e47ce6d832e95c53b6a9142eeaa` | |
| | 8600 | `daa4308f0502c5c6562a9b96a33ab5c80dd051a8a4d61e541f262655be9c62b7` | |
| | 8650 | `38147d42c718faeaf34b861827a2502689aac857c2781a398a018af817176b2b` | |
| | 8700 | `a7849393b913cd9a7ed07602e623110edc3fb12c5f96529f7d639bd26acb96db` | |
| | 8750 | `90a0f8a261839aa615481204c3d7f9286579f50df1f4f8bbb01925ff25e88b2f` | |
| | 8800 | `24b648b0ed510e2958121d1c51c07731b605c8c09a64154a11dcb03ecdc7a191` | |
| | 8850 | `d4a7476c44b7f9ff9dc111acc0dd51c8041c93a31c7555c4a08de46ec1d8bbc5` | |
| | 8900 | `68d0c316249adf620256ef90e5becbddd06920667a3fa858d0028ec2f5d17e7a` | |
| | 8950 | `9c47394c72b08c4da14b32504fdf1eb1711c11fa13d2f49a862379954029ed9c` | |
| | 9000 | `d8ad7edec7e27425f5be4ed0c7d6b1cf32d84ea00e33ebd77fbde00c598dca64` | |
| | 9050 | `cca1ab10de62de41eef5038e87f3b3ffcd17876c2cea284ef9dfd83e6c048028` | |
| | 9100 | `1968f7969b207d9740fdbf4a4efdf0dfebedc17c9fc1b8e7d2cdbc20c43048f5` | |
| | 9150 | `9798249cfdbce9cebb687df9b7cad1c8c87a30ecde243a3ba0853f75007b5bb6` | |
| | 9200 | `9a3a7dd1aaa9a57df1fd68d6a549f0a92e519394e04b5e4371c68200a3aaabab` | |
| | 9250 | `209c1afd3360955e51c4b8fde7ea48789ad62ce2f31f0e839e32627ca6f9fd66` | |
| | 9300 | `4540338c3bad2642b1d8f45ea093fb25ae809f29b520a6787e034dab2995dd45` | |
| | 9350 | `2bb43e35480e82be7b509fcb8cc6b796897c7e1203f50840665f4d23ab7753ba` | |
| | 9400 | `2783ac61aa374ce936ff59c5f3e2df821afefecd774979478d88a3b84e7acad7` | |
| | 9450 | `93601b632ae2e1ae9272ed0a9ae792c0b33b825f867e2dc7b454e8615d6a7cb8` | |
| | 9500 | `14ee41b7b04165ecbc3dea3e1db88aca070df14a85732e21133c17a976db5312` | |
| | 9550 | `4af10fe35c5d087b223b3af96852483aabbc876dda3729601fa0fe3a62ad7fca` | |
| | 9600 | `a5fc866ef4fe196cb959b6a5d9aa34f236bb15fa8294c56da27b3c6b2038e896` | |
| | 9650 | `8f3f58e37fe7b2c51b10e4e8569041f73457d198a1fdf6c53f2b63c38a1e4a28` | |
| | 9700 | `75de6f614ea667790fb4209a8b7ee9fe281cea7b03da9eb307263f4c37b291d3` | |
| | 9750 | `c42785a68715e7dff90e2471aef433a77cf9b01b7c20c323c39a29ef148993c9` | |
| | 9800 | `c9fc88ae437d6c5c0d4532d1225ff3af1e56b2709280f96f864c1695f0552c3d` | |
| | 9850 | `6d7bdbcdb035a8445dc079ee011feac4ead3aeb5eaa15b1950db4f7965b316c8` | |
| | 9900 | `e0bb1cc460798d8eaed3260550a0dd874b53dbbfba6f86c7c66c2ff2c366fa2f` | |
| | 9950 | `b8249f8f20b3a55c992d0ec3ae2d44ad27a93b769cd25926f432dcd611536192` | |
| | 10000 | `37fb6d1893978dfce3a2f87a6bc177e069f4af7ec7b7581249d2c908acfa8ba0` | |
| | 10010 | `04438959d7b317d1a8d972b264e9610a06fd8addde7349c6e9f458ee56dc1e51` | |
|
|
| Steps 10050–13848: see the per-release repositories above (identical files, SHA-256 listed there). |
|
|
| ## File format |
|
|
| `torch.save` dictionary of the trainer: `model`, `opt` (Muon + AdamW), `step`, `config`. Load with |
| `torch.load(path, map_location="cpu", weights_only=True)`. Model class: `train_gpt_ref_r23.py` in the model |
| repositories (1.00–1.06). Every file is 8 860 287 151 bytes. |
|
|
| ## License |
|
|
| Apache-2.0 for the checkpoints. The training data includes CC BY-SA sources (`shared/core_sa`, `shared/legal_sa` of |
| slayer-pl-8x3b); see the dataset card for the licenses of the data. |
|
|
