Hugging Face
Models
Datasets
Spaces
Community
Docs
Enterprise
Pricing
Log In
Sign Up
toksuite
/
tiktoken-gpt-4o
like
0
Follow
TokSuite
7
Text Generation
Transformers
Safetensors
5 languages
llama
toksuite
tokenization
gpt-4o
multilingual
research
robustness
bpe
text-generation-inference
arxiv:
2512.20757
License:
mit
Model card
Files
Files and versions
xet
Community
Deploy
Use this model
main
tiktoken-gpt-4o
/
tokenizer.json
gsaltintas
Upload tokenizer file tiktoken--gpt-4o_vocab.json - Upload model files
6bddde6
verified
6 months ago
raw
Copy download link
history
contribute
delete
Safe
6.62 MB
File too large to display, you can
check the raw version
instead.