File size: 156 Bytes
6e169aa |
1 2 3 4 5 6 7 8 |
{
"vocab_size": 5610,
"pad_token_id": 0,
"bos_token_id": 2,
"eos_token_id": 3,
"unk_token_id": 1,
"tokenizer_class": "PreTrainedTokenizerFast"
} |
6e169aa |
1 2 3 4 5 6 7 8 |
{
"vocab_size": 5610,
"pad_token_id": 0,
"bos_token_id": 2,
"eos_token_id": 3,
"unk_token_id": 1,
"tokenizer_class": "PreTrainedTokenizerFast"
} |