QuantaSparkLabs commited on
Commit
dd541f4
·
verified ·
1 Parent(s): 268ec8c

Delete checkpoint-50/tokenizer_config.json with huggingface_hub

Browse files
checkpoint-50/tokenizer_config.json DELETED
@@ -1,8 +0,0 @@
1
- {
2
- "backend": "tokenizers",
3
- "bos_token": "<|endoftext|>",
4
- "eos_token": "<|endoftext|>",
5
- "model_max_length": 1000000000000000019884624838656,
6
- "pad_token": "<|pad|>",
7
- "tokenizer_class": "TokenizersBackend"
8
- }