Tralalabs
/

Reddit-posts-tokenizer

Model card Files Files and versions

Reddit-posts-tokenizer / tokenizer_config.json

Tralalabs's picture

Update tokenizer_config.json

fce6a63 verified 21 days ago

History Blame Contribute Delete

72 Bytes

	{
	"model_max_length": 2048,
	"tokenizer_class": "GPT2TokenizerFast"
	}