j96-tokenizer-v2 / tokenizer.model

Commit History

25 days ago

  1. Add j96-tokenizer-v2 (SentencePiece Unigram, vocab 96k)

    Verified signature
    2812003