Hugging Face
Models
Datasets
Spaces
Community
Docs
Enterprise
Pricing
Log In
Sign Up
toksuite
/
tokenmonster-englishcode-32000-consistent-v1
like
0
Follow
TokSuite
7
Text Generation
Transformers
Safetensors
5 languages
llama
toksuite
tokenization
tokenmonster
subword
research
robustness
text-generation-inference
arxiv:
2512.20757
License:
mit
Model card
Files
Files and versions
xet
Community
Deploy
Use this model
main
tokenmonster-englishcode-32000-consistent-v1
/
tokenizer.json
gsaltintas
Upload tokenizer file tokenmonster--englishcode-32000-consistent-v1_vocab.json - Upload model files
80b85c8
verified
5 months ago
raw
Copy download link
history
contribute
delete
Safe
684 kB
File too large to display, you can
check the raw version
instead.