omni-tokenizer / tokenizer.json
SurendraVB's picture
Initial release of the 539k Omni-Tokenizer
a987355 verified
Download
History Blame Contribute Delete
94.1 MB
This file is stored with Xet . It is too big to display, but you can still download it.

Xet Pointer Details

( Raw pointer file )
Xet hash:
3e26f3f654be377766da64834dce3f0525e581e5cd752ad74b36cde607333084
Size of remote file:
94.1 MB
·
SHA256:
78cbc81e696620c8146b859d74b4232ca433c7a1fefd5d7b8840a075f979d8c9

Xet efficiently stores Large Files inside Git, intelligently splitting files into unique chunks and accelerating uploads and downloads. More info.