finnianx commited on
Commit
e6a9ded
·
verified ·
1 Parent(s): 9875b5f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -1
README.md CHANGED
@@ -2,10 +2,11 @@
2
  language:
3
  - en
4
  ---
5
- A tiny 55 million parameter trained on 1.3 billion tokens on a custom dataset mixture.
6
  | Dataset | Weight |
7
  |---|---|
8
  | `HuggingFaceFW/fineweb-edu` | 50% |
9
  | `epfml/FineWeb-HQ` | 30% |
10
  | `HuggingFaceTB/cosmopedia` (stories split) | 20% |
 
11
  The tokenizer is a basic bpe tokenizer that was trained on a smaller subset of 80_000 samples of this same data mixture with a vocab size of 8000.
 
2
  language:
3
  - en
4
  ---
5
+ A tiny 55 million parameter trained on 1.3 billion tokens using a custom dataset mixture.
6
  | Dataset | Weight |
7
  |---|---|
8
  | `HuggingFaceFW/fineweb-edu` | 50% |
9
  | `epfml/FineWeb-HQ` | 30% |
10
  | `HuggingFaceTB/cosmopedia` (stories split) | 20% |
11
+ |---|---|
12
  The tokenizer is a basic bpe tokenizer that was trained on a smaller subset of 80_000 samples of this same data mixture with a vocab size of 8000.