{ "tokenizer_class": "PreTrainedTokenizerFast", "model_max_length": 2048, "bos_token": "", "eos_token": "", "unk_token": "", "pad_token": "", "clean_up_tokenization_spaces": false, "added_tokens_decoder": { "0": { "content": "", "special": true, "normalized": false, "lstrip": false, "rstrip": false, "single_word": false }, "1": { "content": "", "special": true, "normalized": false, "lstrip": false, "rstrip": false, "single_word": false }, "2": { "content": "", "special": true, "normalized": false, "lstrip": false, "rstrip": false, "single_word": false }, "3": { "content": "", "special": true, "normalized": false, "lstrip": false, "rstrip": false, "single_word": false } }, "chat_template": "{#-\n OpenSI-Haiku 对话模板(与 src/chat_format.py 逐字符等价,由 self_check 保证)。\n\n 要点:\n * 不引入新 special token,role 用纯文本标记 ### System/User/Assistant。\n * 助手轮以 eos_token() 收尾——它就是文档边界,模型据此学会「说完」。\n * add_generation_prompt=True 时以「### Assistant:\\n」结尾,等待模型生成。\n * 所有模板字面空白都用 {%- -%} 去掉,实际文本完全由表达式产出,\n 这样才会与 Python 的字符串拼接逐字符一致。\n-#}\n{%- for message in messages -%}\n{%- if message['role'] == 'system' -%}\n{{ '### System:\\n' + message['content'] + '\\n' }}\n{%- elif message['role'] == 'user' -%}\n{{ '### User:\\n' + message['content'] + '\\n' }}\n{%- elif message['role'] == 'assistant' -%}\n{{ '### Assistant:\\n' + message['content'] + eos_token }}\n{%- endif -%}\n{%- endfor -%}\n{%- if add_generation_prompt -%}\n{{ '### Assistant:\\n' }}\n{%- endif -%}\n" }