Hugging Face's logo

TheBloke
/
zephyr-7B-beta-AWQ

Text Generation
Transformers
Safetensors
English
mistral
Generated from Trainer
conversational
text-generation-inference
4-bit precision
awq
Model card Files
xet
Community
3
New discussion

vLLM out of memory

2
#2 opened over 2 years ago by
cfrancois7

Is there a way to finetune these AWQ beasts?

4
#1 opened over 2 years ago by
yasmolin