TaoNet-mini-A2 / tokenizer_config.json
Lobakkang's picture
Upload folder using huggingface_hub
14531a0 verified
Raw
History Blame Contribute Delete
322 Bytes
{
"backend": "custom",
"bos_token": "<BOS>",
"eos_token": "<EOS>",
"unk_token": "<UNK>",
"pad_token": "<PAD>",
"tokenizer_class": "TaoNetTokenizer",
"model_max_length": 1000000000000000019884624838656,
"extra_special_tokens": [
"\n",
"<think>",
"<user>",
"<assistant>",
"<image>"
]
}