marlow/tipsv2-b14-bucket / tokenizer_config.json
marlow's picture
download
raw
292 Bytes
{
"backend": "tokenizers",
"bos_token": null,
"do_lower_case": true,
"eos_token": null,
"model_max_length": 64,
"pad_token": "<pad>",
"processor_class": "Tipsv2Processor",
"token_type_ids_pattern": "all_zeros",
"tokenizer_class": "Tipsv2Tokenizer",
"unk_token": "<unk>"
}

Xet Storage Details

Size:
292 Bytes
·
Xet hash:
41adf5205e006e61ba8404029624ddb9f9d036ad738544766ee398134fccec3a

Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.