Commit History

Update tokenizer_class to PreTrainedTokenizerFast for GGUF compatibility
dab7200
verified

spitfire4794 commited on

Add standard contractions to pre-tokenizer Regex for GGUF compatibility
feceaf6
verified

spitfire4794 commited on

Update tokenizer_class to LlamaTokenizerFast for GGUF compatibility
6ccfa3b
verified

spitfire4794 commited on

Squash history to single initial commit
616f6b1

spitfire4794 commited on