Hugging Face
Models
Datasets
Spaces
Community
Docs
Enterprise
Pricing
Log In
Sign Up
isaacus
/
kanon-2-tokenizer
like
0
Follow
Isaacus
29
English
legal
License:
apache-2.0
Model card
Files
Files and versions
xet
Community
main
kanon-2-tokenizer
6.16 MB
1 contributor
History:
6 commits
umarbutler
docs: rewording
2e8da5c
verified
3 months ago
.gitattributes
1.52 kB
initial commit
7 months ago
README.md
190 Bytes
docs: rewording
3 months ago
merges.txt
495 kB
Added Kanon 2's tokenizer.
7 months ago
special_tokens_map.json
1.18 kB
Added Kanon 2's tokenizer.
7 months ago
tiktoken_tokenizer.json
980 kB
Added Kanon 2's tokenizer.
7 months ago
tokenizer.json
3.64 MB
feat: increase context window
4 months ago
tokenizer_config.json
1.34 kB
feat: increase context window
3 months ago
vocab.json
1.04 MB
Added Kanon 2's tokenizer.
7 months ago
