beng_bpe / tokenizer_config.json

Commit History

Update tokenizer_class to PreTrainedTokenizerFast for GGUF compatibility
5d34c66
verified

spitfire4794 commited on

Update tokenizer_class to LlamaTokenizerFast for GGUF compatibility
db101bf
verified

spitfire4794 commited on

Upload 4 files
c2b3af0
verified

spitfire4794 commited on