aaaaaaaa / tokenizer /tokenizer_config.json
karrrr123456's picture
Upload 9 files
1952869 verified
raw
history blame contribute delete
52 Bytes
{
"tokenizer_type": "BPE",
"vocab_size": 30000
}