Models
Datasets
Spaces
Buckets new
Docs
Enterprise
Pricing
Log In
Sign Up

isaacus
/

kanon-2-tokenizer

Model card Files Files and versions

kanon-2-tokenizer

4.71 MB

Ctrl+K

Ctrl+K

1 contributor

History: 2 commits

umarbutler's picture

Added Kanon 2's tokenizer.

d5c6acb verified 10 months ago

.gitattributes

1.52 kB
initial commit 10 months ago
README.md

31 Bytes
initial commit 10 months ago
merges.txt

495 kB
Added Kanon 2's tokenizer. 10 months ago
special_tokens_map.json

1.18 kB
Added Kanon 2's tokenizer. 10 months ago
tiktoken_tokenizer.json

980 kB
Added Kanon 2's tokenizer. 10 months ago
tokenizer.json

2.19 MB
Added Kanon 2's tokenizer. 10 months ago
tokenizer_config.json

1.34 kB
Added Kanon 2's tokenizer. 10 months ago
vocab.json

1.04 MB
Added Kanon 2's tokenizer. 10 months ago