Update tokenizer_class to PreTrainedTokenizerFast for GGUF compatibility dab7200 verified spitfire4794 commited on Jul 11
Add standard contractions to pre-tokenizer Regex for GGUF compatibility feceaf6 verified spitfire4794 commited on Jul 11
Update tokenizer_class to LlamaTokenizerFast for GGUF compatibility 6ccfa3b verified spitfire4794 commited on Jul 11