test / tokenizer /understanding_yap.json
WICKED4950's picture
Upload tokenizer/understanding_yap.json with huggingface_hub
1be653b verified
{
"name": "test",
"vocab_size": 0,
"special_tokens": [
"<|EOS|>",
"<|UK|>"
],
"eos_token": "<|EOS|>",
"uk_token": "<|UK|>"
}