Lens-Turbo / tokenizer /special_tokens_map.json
SceneWorks's picture
Upload folder using huggingface_hub
33acd26 verified
Raw
History Blame Contribute Delete
98 Bytes
{
"bos_token": "<|startoftext|>",
"eos_token": "<|return|>",
"pad_token": "<|endoftext|>"
}