alisawuffles
/

superbpe-tokenizer-128k

Model card Files Files and versions

superbpe-tokenizer-128k / added_tokens.json

alisawuffles's picture

Upload folder using huggingface_hub

f5530c9 verified 6 months ago

history blame contribute delete

30 Bytes

	{
	"<\|endoftext\|>": 128000
	}