tiny-random-cogview4 / tokenizer /added_tokens.json
a-r-r-o-w's picture
Upload folder using huggingface_hub
0992999 verified
raw
history blame contribute delete
356 Bytes
{
"<eop>": 151334,
"<sop>": 151333,
"<|assistant|>": 151337,
"<|begin_of_image|>": 151339,
"<|begin_of_video|>": 151341,
"<|end_of_image|>": 151340,
"<|end_of_video|>": 151342,
"<|endoftext|>": 151329,
"<|observation|>": 151338,
"<|system|>": 151335,
"<|user|>": 151336,
"[MASK]": 151330,
"[gMASK]": 151331,
"[sMASK]": 151332
}