Hugging Face's logo

isaacus
/
kanon-2-tokenizer

English
legal
Model card Files
xet
 Community
kanon-2-tokenizer

Commit History

  1. fix: drop `special_tokens_map.json` to fix transformers v5

    umarbutler
    Verified signature
    c53b842

  1. docs: rewording

    umarbutler
    Verified signature
    2e8da5c

  2. docs: add README

    umarbutler
    Verified signature
    b3c05d9

  3. feat: increase context window

    umarbutler
    Verified signature
    f213a8b

  1. feat: increase context window

    umarbutler
    Verified signature
    e458fa8

  1. Added Kanon 2's tokenizer.

    umarbutler
    Verified signature
    d5c6acb

  2. initial commit

    umarbutler
    Verified signature
    3f61691