codellm-125m / tokenizer /added_tokens.json
devoppro's picture
Upload folder using huggingface_hub
c4c3ef4 verified
{
"<|cpp|>": 50265,
"<|fim_middle|>": 50258,
"<|fim_prefix|>": 50257,
"<|fim_suffix|>": 50259,
"<|go|>": 50263,
"<|javascript|>": 50261,
"<|python|>": 50260,
"<|rust|>": 50262,
"<|typescript|>": 50264
}