code-search-net-tokenizer / added_tokens.json
rain1898's picture
Upload tokenizer
483938b
{
"<|endoftext|>": 0
}