j48-tokenizer-v2 / tokenizer.model
eightman999's picture
Add j48-tokenizer-v2 (SentencePiece Unigram, vocab 48k)
ed70cd3 verified
History Blame Contribute Delete
951 kB
This file is stored with Xet . It is too big to display, but you can still download it.

Xet Pointer Details

( Raw pointer file )
Xet hash:
b82e54396d8cc31e1150c7d8fa901af613bda9c6ea2608ecbb30d92a59788043
Size of remote file:
951 kB
·
SHA256:
11d0d24290685b9102e6f61728eafbb8617899ac626358d0a4d3f32af0cda572

Xet efficiently stores Large Files inside Git, intelligently splitting files into unique chunks and accelerating uploads and downloads. More info.