smoltokenizer / tokenizer.json
eliebak's picture
eliebak HF Staff
commit of fast smoltokenizer
fddfcaa
This file is stored with Xet . It is too big to display, but you can still download it.

Large File Pointer Details

( Raw pointer file )
SHA256:
a5078c05e94e5980118e913ed49426e7b1cdf93dcc6ccefa7967fafd2f7e255e
Pointer size:
129 Bytes
·
Size of remote file:
6.17 kB
·
Xet hash:
f4f0f0a5b66f6d171b005dfbbddf92f4e798fcf389472ff90e52002290aeb86c

Xet efficiently stores Large Files inside Git, intelligently splitting files into unique chunks and accelerating uploads and downloads. More info.