fastai
English
causal_lm
Aika / tokenizer_config.json
Amploud's picture
Upload 5 files
191a5cb verified
{
"unk_token": "<unk>",
"sep_token": "</s>",
"pad_token": "<pad>",
"cls_token": "<s>",
"mask_token": "<mask>",
"model_max_length": 2048,
"do_lower_case": false
}