mns_tokenizer / added_tokens.json
davaa33's picture
Upload tokenizer
378f59b verified
raw
history blame contribute delete
72 Bytes
{
"<pad>": 8002,
"[CLS]": 8000,
"[MASK]": 8003,
"[SEP]": 8001
}