llm_100M / tokenizer_config.json
angkul07's picture
Update tokenizer_config.json
9bea6a2 verified
raw
history blame contribute delete
72 Bytes
{
"tokenizer_class": "LlamaTokenizer",
"model_max_length": 256
}