AAichael/Amuse-bucket / tokenizer /special_tokens_map.json
AAichael's picture
download
raw
496 Bytes
{
"bos_token": {
"content": "<|startoftext|>",
"lstrip": false,
"normalized": true,
"rstrip": false,
"single_word": false
},
"eos_token": {
"content": "<|endoftext|>",
"lstrip": false,
"normalized": true,
"rstrip": false,
"single_word": false
},
"pad_token": "<|endoftext|>",
"unk_token": {
"content": "<|endoftext|>",
"lstrip": false,
"normalized": true,
"rstrip": false,
"single_word": false
}
}

Xet Storage Details

Size:
496 Bytes
·
Xet hash:
7605d0474f79ca84d5534267e15f8d6600ecedd0d79abd33702cfcd433bec0bb

Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.