Set tokenizer class to `PreTrainedTokenizerFast` 976ad55 verified Xenova HF Staff commited on Jul 19, 2024
Set `clean_up_tokenization_spaces` to false 6a4dc9f verified Xenova HF Staff commited on Jul 19, 2024
Do not add eos token by default (#8) 4f81d78 verified patrickvonplaten Xenova HF Staff commited on Jul 18, 2024
Add note to install transformers from source (#3) 8b37f73 verified patrickvonplaten Xenova HF Staff commited on Jul 18, 2024
Upload consolidated.safetensors with huggingface_hub f5467e7 verified patrickvonplaten commited on Jul 17, 2024
