StenToken / Stentoken /EnglishCode /8k /sten_tokenizer.pt
StentorLabs's picture
Publish runtime tokenizer bundles from 4K through 32K
0ea5ca4 verified
Download
History Blame Contribute Delete
586 kB
This file is stored with Xet . It is too big to display, but you can still download it.

Xet Pointer Details

( Raw pointer file )
Xet hash:
4aed3c2c9fd9c18f40a160b1408781e1c17a885c67919c67ff565d23e4af8c53
Size of remote file:
586 kB
·
SHA256:
4bbb3f382b60c15d64dc3bb741ab8d675707ff54983b16b8339a45b18b0dd312

Xet efficiently stores Large Files inside Git, intelligently splitting files into unique chunks and accelerating uploads and downloads. More info.