Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -1,8 +1,8 @@
|
|
| 1 |
---
|
| 2 |
library_name: pytorch
|
| 3 |
---
|
| 4 |
-
# Slayer 11B pilot
|
| 5 |
|
| 6 |
Private backup of an experimental approximately 11.18B-parameter model trained from scratch on a subset of SlayerLab/slayer-pl-8x3b. This is an early pilot, not a finished or evaluated model.
|
| 7 |
|
| 8 |
-
`
|
|
|
|
| 1 |
---
|
| 2 |
library_name: pytorch
|
| 3 |
---
|
| 4 |
+
# Slayer 11B pilot weights
|
| 5 |
|
| 6 |
Private backup of an experimental approximately 11.18B-parameter model trained from scratch on a subset of SlayerLab/slayer-pl-8x3b. This is an early pilot, not a finished or evaluated model.
|
| 7 |
|
| 8 |
+
`weights-only.pt` is a PyTorch dictionary containing `model`, `config`, and `step`. It preserves the original weight precision, but does not contain the optimizer state and cannot resume the original training state exactly. This uses the custom `train_gpt.py` architecture, not Transformers format; the reference tokenizer is SlayerLab/Slayer1B-1.00. Load PyTorch files only from trusted sources. The file appears after export and upload finish.
|