Models
Datasets
Spaces
Buckets new
Docs
Enterprise
Pricing
Log In
Sign Up

isaacus
/

kanon-2-tokenizer

Model card Files Files and versions

kanon-2-tokenizer

6.16 MB

Ctrl+K

Ctrl+K

1 contributor

History: 6 commits

umarbutler's picture

docs: rewording

2e8da5c verified 6 months ago

.gitattributes

1.52 kB
initial commit 10 months ago
README.md

190 Bytes
docs: rewording 6 months ago
merges.txt

495 kB
Added Kanon 2's tokenizer. 10 months ago
special_tokens_map.json

1.18 kB
Added Kanon 2's tokenizer. 10 months ago
tiktoken_tokenizer.json

980 kB
Added Kanon 2's tokenizer. 10 months ago
tokenizer.json

3.64 MB
feat: increase context window 7 months ago
tokenizer_config.json

1.34 kB
feat: increase context window 6 months ago
vocab.json

1.04 MB
Added Kanon 2's tokenizer. 10 months ago