Update model card for log-spaced checkpoints
Browse files
README.md
CHANGED
|
@@ -28,14 +28,17 @@ Each branch contains a full training checkpoint at a given step, including:
|
|
| 28 |
|
| 29 |
## Branches
|
| 30 |
|
| 31 |
-
|
| 32 |
|
| 33 |
- `step0` — initialization
|
| 34 |
- `step{1,2,4,8,16,32,64,128,256,512}` — log-spaced early checkpoints
|
| 35 |
-
- `
|
|
|
|
| 36 |
|
| 37 |
Branch `step143000` corresponds to the final model.
|
| 38 |
|
|
|
|
|
|
|
| 39 |
## Converting to HuggingFace Format
|
| 40 |
|
| 41 |
To convert a checkpoint to HuggingFace Transformers format, use the conversion script from [GPT-NeoX](https://github.com/EleutherAI/gpt-neox):
|
|
|
|
| 28 |
|
| 29 |
## Branches
|
| 30 |
|
| 31 |
+
20 log-spaced checkpoints are available as branches:
|
| 32 |
|
| 33 |
- `step0` — initialization
|
| 34 |
- `step{1,2,4,8,16,32,64,128,256,512}` — log-spaced early checkpoints
|
| 35 |
+
- `step{1000,2000,4000,8000,16000,32000,64000,128000}` — log-spaced training checkpoints
|
| 36 |
+
- `step143000` — final model
|
| 37 |
|
| 38 |
Branch `step143000` corresponds to the final model.
|
| 39 |
|
| 40 |
+
> **Note:** To keep storage requirements manageable, this repository provides a log-spaced subset of 20 checkpoints rather than all 154 training checkpoints. If you need linearly-spaced checkpoints (every 1,000 steps), the HuggingFace Transformers-compatible weights for all 154 checkpoints are available at [`EleutherAI/pythia-2.8b`](https://huggingface.co/EleutherAI/pythia-2.8b).
|
| 41 |
+
|
| 42 |
## Converting to HuggingFace Format
|
| 43 |
|
| 44 |
To convert a checkpoint to HuggingFace Transformers format, use the conversion script from [GPT-NeoX](https://github.com/EleutherAI/gpt-neox):
|