OddEvenDumb-Fast / tokenizer_config.json
56m's picture
Upload 8 files
199efb6 verified
Raw
History Blame Contribute Delete
179 Bytes
{
"add_bos_token": true,
"add_prefix_space": false,
"bos_token": "<BOS>",
"eos_token": "<EOS>",
"model_max_length": 512,
"tokenizer_class": "PreTrainedTokenizerFast"
}