Hugging Face's logo

flexitok
/
bpe_script_SEAS_16000

Indonesian
Vietnamese
tokenizer
bpe
flexitok
fineweb2
Model card Files
xet
 Community
bpe_script_SEAS_16000
File size: 82 Bytes
3779e6b
 
 
 
1
2
3
4
5
data:
  tokenizer:
    name: huggingface
    path: flexitok/bpe_script_SEAS_16000