tokenizer-128k / tokenizer.model
timpal0l's picture
Upload 128k vocab SentencePiece BPE tokenizer (trained on 175GB)
43f50d5 verified
This file is stored with Xet . It is too big to display, but you can still download it.

Large File Pointer Details

( Raw pointer file )
SHA256:
e8e8b94d7914eee2a71dc2d471334b6256ddabd7962b633e108a442d3626f88c
Pointer size:
132 Bytes
·
Size of remote file:
2.35 MB
·
Xet hash:
a4eb641d945b689ff622554b74a25ce0abb8615198d178d418e470d7da20adfb

Xet efficiently stores Large Files inside Git, intelligently splitting files into unique chunks and accelerating uploads and downloads. More info.