Hugging Face
Models
Datasets
Spaces
Community
Docs
Enterprise
Pricing
Log In
Sign Up
styalai
/
tokenizer5000
like
0
Transformers
arxiv:
1910.09700
Model card
Files
Files and versions
xet
Community
Deploy
Use this model
main
tokenizer5000
/
added_tokens.json
styalai
Upload tokenizer
5e5a01b
verified
over 1 year ago
raw
Copy download link
history
blame
contribute
delete
Safe
86 Bytes
{
"</s>"
:
3648
,
"<mask>"
:
3651
,
"<pad>"
:
3650
,
"<s>"
:
3647
,
"<unk>"
:
3649
}