again-resized / tokenizer.json
pranay1706's picture
added more new tokens
ecafafe verified
This file is stored with Xet . It is too big to display, but you can still download it.

Large File Pointer Details

( Raw pointer file )
SHA256:
2480437aa24e5c6f3db47c12ea5e9af4ab03994b37792e071e2bc6f72da21f6d
Pointer size:
133 Bytes
·
Size of remote file:
15.5 MB
·
Xet hash:
aef7a9c9fda41cb0b7cde3ba232b9de61fb16b1054fe3afc9390abfec017ca01

Xet efficiently stores Large Files inside Git, intelligently splitting files into unique chunks and accelerating uploads and downloads. More info.