j32-tokenizer-v2 / tokenizer.model
eightman999's picture
Add j32-tokenizer-v2 (SentencePiece Unigram, vocab 32k)
e755e4f verified
History Blame Contribute Delete
606 kB
This file is stored with Xet . It is too big to display, but you can still download it.

Xet Pointer Details

( Raw pointer file )
Xet hash:
2aef7d5917fb36ed76209e9acf616ae982bbf67672a2bfb895cc3f37f2ad1c8d
Size of remote file:
606 kB
·
SHA256:
23b0e6dc7d1cdc5c8e6e3f0a58bc855b848a306dcd7e6b0447808a7d66d6e088

Xet efficiently stores Large Files inside Git, intelligently splitting files into unique chunks and accelerating uploads and downloads. More info.