Danderlin/data2 / models /Yntec /IdleFancy /tokenizer /special_tokens_map.json
Danderlin's picture
download
raw
586 Bytes
{
"bos_token": {
"content": "<|startoftext|>",
"lstrip": false,
"normalized": true,
"rstrip": false,
"single_word": false
},
"eos_token": {
"content": "<|endoftext|>",
"lstrip": false,
"normalized": true,
"rstrip": false,
"single_word": false
},
"pad_token": {
"content": "<|endoftext|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"unk_token": {
"content": "<|endoftext|>",
"lstrip": false,
"normalized": true,
"rstrip": false,
"single_word": false
}
}

Xet Storage Details

Size:
586 Bytes
·
Xet hash:
83c5abcbee8bfd356b16a3fb53e237ea271adeb0d4fb5be7c16ba816d564edda

Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.