Models
Datasets
Spaces
Buckets new
Docs
Enterprise
Pricing
Log In
Sign Up

Corianas
/

char128_shift_tokenizer

Model card Files Files and versions

char128_shift_tokenizer

19.7 kB

Ctrl+K

Ctrl+K

1 contributor

History: 5 commits

Corianas's picture

Update README.md

4281937 verified 8 months ago

src
Upload build_shift_char_tokenizer.py 8 months ago
.gitattributes

1.52 kB
initial commit 8 months ago
README.md

4.06 kB
Update README.md 8 months ago
special_tokens_map.json

105 Bytes
Upload 3 files 8 months ago
tokenizer.json

7.01 kB
Upload 3 files 8 months ago
tokenizer_config.json

1.01 kB
Upload 3 files 8 months ago