Hugging Face's logo

flexitok
/
bpe_cmn_Hani_8000

Mandarin Chinese
tokenizer
bpe
flexitok
fineweb2
Model card Files
xet
Community
bpe_cmn_Hani_8000
1.1 MB
  • 1 contributor
 History: 8 commits
gsaltintas's picture
gsaltintas
Upload fineweb2_hq/flexitok--bpe_cmn_Hani_8000_overlap.json with huggingface_hub
dc85a41 verified