morginalium
/

tinystories-tokenizers

Model card Files Files and versions

tinystories-tokenizers / tokenizer_50257.json

morginalium's picture

Add full suite of BPE tokenizers (1k to 50k)

da47eb4 verified about 1 month ago

History Contribute Delete

3.59 MB

File too large to display, you can check the raw version instead.