weiser
/

124M-0.6

Text Generation

text-generation-inference

Model card Files Files and versions

Ambuj Varshney commited on Nov 10, 2024

Commit

b16ed8f

·

verified ·

1 Parent(s): 14a97da

typo

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -20,7 +20,7 @@ This repository hosts a small language model developed as part of the TinyLLM fr
 - **Parameters:** 124M (Hidden Size = 768)
 - **Architecture:** Decoder-only transformer
-- **Training Data:** Up to 10B tokens from the [SHL](http://www.shl-dataset.org/) and [Fineweb](https://huggingface.co/datasets/HuggingFaceFW/fineweb) datasets, combined in a 0.6 ratio
 - **Input and Output Modality:** Text
 - **Context Length:** 1024

 - **Parameters:** 124M (Hidden Size = 768)
 - **Architecture:** Decoder-only transformer
+- **Training Data:** Up to 10B tokens from the [SHL](http://www.shl-dataset.org/) and [Fineweb](https://huggingface.co/datasets/HuggingFaceFW/fineweb) datasets, combined in a 6:4 ratio
 - **Input and Output Modality:** Text
 - **Context Length:** 1024