Nano-H / tokenizer_config.json
Fu01978's picture
Upload folder using huggingface_hub
1f46d9d verified
{
"backend": "tokenizers",
"model_max_length": 1000000000000000019884624838656,
"tokenizer_class": "TokenizersBackend"
}