Update README.md
Browse files
README.md
CHANGED
|
@@ -9,6 +9,7 @@ tags:
|
|
| 9 |
- instruction-tuned
|
| 10 |
- sft
|
| 11 |
datasets:
|
|
|
|
| 12 |
- roneneldan/TinyStoriesInstruct
|
| 13 |
pipeline_tag: text-generation
|
| 14 |
---
|
|
@@ -22,8 +23,6 @@ A 30M parameter causal language model fine-tuned for instruction-following on th
|
|
| 22 |
- **Parameters**: ~30M
|
| 23 |
- **Architecture**: LLaMA-style transformer
|
| 24 |
- **Base Model**: TinyStories 30M pre-trained
|
| 25 |
-
- **Fine-tuning**: SFT on TinyStoriesInstruct (~117K examples, ~4 epochs)
|
| 26 |
-
- **Training Time**: ~2 hours on single GPU
|
| 27 |
|
| 28 |
## Usage
|
| 29 |
|
|
|
|
| 9 |
- instruction-tuned
|
| 10 |
- sft
|
| 11 |
datasets:
|
| 12 |
+
- ltg/babylm-2024-baby-cosmo-fine-10m
|
| 13 |
- roneneldan/TinyStoriesInstruct
|
| 14 |
pipeline_tag: text-generation
|
| 15 |
---
|
|
|
|
| 23 |
- **Parameters**: ~30M
|
| 24 |
- **Architecture**: LLaMA-style transformer
|
| 25 |
- **Base Model**: TinyStories 30M pre-trained
|
|
|
|
|
|
|
| 26 |
|
| 27 |
## Usage
|
| 28 |
|