Hugging Face's logo

j-llm
/
j128-tokenizer-v2

Japanese
sentencepiece
tokenizer
unigram
japanese
Model card Files
xet
 Community
j128-tokenizer-v2 / tokenizer.model
eightman999's picture
eightman999
Add j128-tokenizer-v2 (SentencePiece Unigram, vocab 128k)
0ef6fc2 verified
History Blame Contribute Delete
2.88 MB
This file is stored with Xet . It is too big to display, but you can still download it.

Xet Pointer Details

( Raw pointer file )
Xet hash:
4e2de5d2ea6c6bb1b34d316421aa45e82d63a3957a30e1cd1609c200fa34b815
Size of remote file:
2.88 MB
·
SHA256:
5ae4cba94faf948a32021afb6aedabb0a78536dfdc5b946f2ddfcaf9e00f28f0

Xet efficiently stores Large Files inside Git, intelligently splitting files into unique chunks and accelerating uploads and downloads. More info.