YAML Metadata Warning: empty or missing yaml metadata in repo card
Check out the documentation for more information.
Should be working properly with KoboldCPP and other stuff. I changed EOS token to <|eot_id|> and this prevents endless generation.
I also converted original weights back to fp32 before quantization.
Edit: Highly recommending you to download more recent quants. 17/05/2024
- Downloads last month
- -
Hardware compatibility
Log In to add your hardware
6-bit
8-bit
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support