Hugging Face
Models
Datasets
Spaces
Community
Docs
Enterprise
Pricing
Log In
Sign Up
Corianas
/
char128_shift_tokenizer
like
0
English
License:
mit
Model card
Files
Files and versions
xet
Community
main
char128_shift_tokenizer
19.5 kB
1 contributor
History:
7 commits
Corianas
Update tokenizer.json
c1010b0
verified
3 months ago
src
Upload build_shift_char_tokenizer.py
5 months ago
.gitattributes
1.52 kB
initial commit
5 months ago
README.md
3.85 kB
Update README.md
3 months ago
special_tokens_map.json
105 Bytes
Upload 3 files
5 months ago
tokenizer.json
7.01 kB
Update tokenizer.json
3 months ago
tokenizer_config.json
1.01 kB
Upload 3 files
5 months ago