ResearchModel-Rivulet-s25 / tokenizer_config.json
123sg's picture
Add tokenizer_config.json from checkpoint_4214 release package
e95d3c2 verified
Raw History Blame Contribute Delete
79 Bytes
{
"model_max_length": 4121,
"tokenizer_class": "PreTrainedTokenizerFast"
}