tokenizer-dynamic-include-template / tokenizer_config.json
izm1chael's picture
Publish Layerfault deep challenge LF-CH-TOKX-0008
1fd8df4 verified
Raw
History Blame Contribute Delete
149 Bytes
{
"bos_token": "<bos>",
"chat_template": "{% include template_name %}",
"eos_token": "<eos>",
"tokenizer_class": "PreTrainedTokenizerFast"
}