j128-tokenizer-v2 / tokenizer.model

Commit History

30 days ago

  1. Add j128-tokenizer-v2 (SentencePiece Unigram, vocab 128k)

    Verified signature
    0ef6fc2