wikitext-2-raw-v1 / special_tokens_map.json
goabonga's picture
Upload tokenizer files (vocab, config, README)
6dee153 verified
raw
history blame contribute delete
98 Bytes
{
"pad_token": "<pad>",
"unk_token": "<unk>",
"bos_token": "<bos>",
"eos_token": "<eos>"
}