Hugging Face's logo

j-llm
/
j64-tokenizer-v2

Japanese
sentencepiece
tokenizer
unigram
japanese
Model card Files
xet
 Community
j64-tokenizer-v2 / tokenizer.model
eightman999's picture
eightman999
Add j64-tokenizer-v2 (SentencePiece Unigram, vocab 64k)
e6bbd1f verified
History Blame Contribute Delete
1.31 MB
This file is stored with Xet . It is too big to display, but you can still download it.

Xet Pointer Details

( Raw pointer file )
Xet hash:
753f475d95522a7b269a5e538dedcc787932a9f03fad7bf79fbffe0ba058c8b4
Size of remote file:
1.31 MB
·
SHA256:
0f2fefec0cd5f84a4efa76f31ebcacd0510b341219bf6037e1b49ce0d84a0a66

Xet efficiently stores Large Files inside Git, intelligently splitting files into unique chunks and accelerating uploads and downloads. More info.