masa-research
/

example-custom-tokenizer

Model card Files Files and versions

example-custom-tokenizer / added_tokens.json

Masa-Erland's picture

Upload tokenizer

df13113 verified almost 2 years ago

history blame contribute delete

34 Bytes

	{
	"<\|file_separator\|>": 32000
	}