cclark
/

cbr-rnn

model_hub_mixin

pytorch_model_hub_mixin

Model card Files Files and versions

cbr-rnn / tokenizer_config.json

cclark's picture

Upload tokenizer

0aa38b8 verified about 1 month ago

History Blame Contribute Delete

175 Bytes

	{
	"backend": "tokenizers",
	"bos_token": "<BOS>",
	"eos_token": "<EOS>",
	"model_max_length": 1000000000000000019884624838656,
	"tokenizer_class": "TokenizersBackend"
	}