kylelovesllms
/

my-tokenizer

Model card Files Files and versions

my-tokenizer / special_tokens_map.json

kylelovesllms's picture

Upload tokenizer

6bdfc3c verified about 1 year ago

history blame contribute delete

85 Bytes

{"bos_token": "<s>", "eos_token": "</s>", "unk_token": "[UNK]", "pad_token": "[PAD]"}