SurjoLabs
/

surjo_tokenizer

Model card Files Files and versions

surjo_tokenizer / special_tokens_map.json

spitfire4794's picture

Upload folder using huggingface_hub

456f991 verified 12 days ago

History Blame Contribute Delete

134 Bytes

	{
	"bos_token": "<\|bos\|>",
	"eos_token": "<\|eos\|>",
	"unk_token": "<\|unk\|>",
	"pad_token": "<\|pad\|>",
	"mask_token": "<\|mask\|>"
	}