j64-tokenizer-v2 / tokenizer.model

Commit History

24 days ago

  1. Add j64-tokenizer-v2 (SentencePiece Unigram, vocab 64k)

    Verified signature
    e6bbd1f