Update model card for checkpoints/checkpoint-10B
Browse files
README.md
CHANGED
|
@@ -56,7 +56,7 @@ model repository is intended to be used with the standalone Apache-2.0 runtime.
|
|
| 56 |
- Candidate: Kaczmarz-normalized GDN-2 gates
|
| 57 |
- Training data source: FineWeb-Edu `sample/100BT` local parquet
|
| 58 |
- Token budget for this run: 10B
|
| 59 |
-
- Current milestone:
|
| 60 |
- Sequence length: 4096 tokens
|
| 61 |
- Global batch tokens: 1,048,576
|
| 62 |
- Tokenizer: `TinyLlama/TinyLlama_v1.1`
|
|
|
|
| 56 |
- Candidate: Kaczmarz-normalized GDN-2 gates
|
| 57 |
- Training data source: FineWeb-Edu `sample/100BT` local parquet
|
| 58 |
- Token budget for this run: 10B
|
| 59 |
+
- Current milestone: 10,000,000,000 tokens
|
| 60 |
- Sequence length: 4096 tokens
|
| 61 |
- Global batch tokens: 1,048,576
|
| 62 |
- Tokenizer: `TinyLlama/TinyLlama_v1.1`
|