Hunyuen-preview2 / special_tokens_map.json
codys12's picture
Upload tokenizer
e78e089 verified
{
"additional_special_tokens": [
"<|startoftext|>",
"<|extra_0|>",
"<|extra_4|>",
"<|extra_5|>",
"<|eos|>"
],
"eos_token": "<|eos|>",
"pad_token": "<|pad|>"
}