Hugging Face's logo

ByteSpanTokenisers
/
tokenizers

Model card Files
xet
Community
codebyzeb commited on
Commit
5bdef6c
·
verified ·
1 Parent(s): 9c3058f

Upload folder using huggingface_hub

Browse files
Files changed (4) hide show
  1. frequencymulti_256000/merges.txt +0 -0
  2. frequencymulti_256000/merges_data.csv +0 -0
  3. frequencymulti_256000/tokenizer.json +2 -2
  4. frequencymulti_256000/vocab.json +0 -0
frequencymulti_256000/merges.txt CHANGED
The diff for this file is too large to render. See raw diff
 
frequencymulti_256000/merges_data.csv CHANGED
The diff for this file is too large to render. See raw diff
 
frequencymulti_256000/tokenizer.json CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:8c1ac462fae29c81b13d1ee24e1eedfe812d9c0d374ff3c61ba636cebaf52fd7
3
- size 20274158
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2b661f27482b157ee9aa4b9848ae58b9f6204d3a1ee16d1543fd02e2e7574491
3
+ size 19953174
frequencymulti_256000/vocab.json CHANGED
The diff for this file is too large to render. See raw diff