tokenizer-special-token-reassignment / tokenizer_config.json
izm1chael's picture
Publish Layerfault deep challenge LF-CH-TOKX-0010
9cf85dd verified
Raw
History Blame Contribute Delete
214 Bytes
{
"bos_token": "<bos>",
"chat_template": "{% for message in messages %}{{ message[\"role\"] }}: {{ message[\"content\"] }}{% endfor %}",
"eos_token": "<bos>",
"tokenizer_class": "PreTrainedTokenizerFast"
}