Hugging Face's logo

flexitok
/
bpe_jpn_Jpan_16000

Japanese
tokenizer
bpe
flexitok
fineweb2
Model card Files
xet
Community
bpe_jpn_Jpan_16000
2.35 kB
  • 1 contributor
 History: 4 commits
gsaltintas's picture
gsaltintas
Upload folder using huggingface_hub
7b19914 verified