TinyStories-Regional
/

tokenizer-comparisons

Model card Files Files and versions

tokenizer-comparisons / Tiktoken

1.74 GB

2 contributors

History: 4 commits

Jinom's picture

Upload folder using huggingface_hub

6b91b07 verified 12 months ago

ckpt_tiktoken-beng_val=0.135.pt
542 MB
xet

Upload folder using huggingface_hub 12 months ago
ckpt_tiktoken-hindi_val=0.149.pt
651 MB
xet

Upload folder using huggingface_hub 12 months ago
ckpt_tiktoken-marathi_val=0.167.pt
542 MB
xet

Upload folder using huggingface_hub 12 months ago
test.txt
0 Bytes

Create Tiktoken/test.txt 12 months ago