Supporting GPTQ/AWQ
#2
by fzyzcjy - opened
Hi thank you for this GGUF! I would like to run it in vllm, thus I wonder whether there will be GPTQ/AWQ quantization in the future?
EDIT: Oh I see it seems you only quantize to format GGUF... Then I wonder whether there will be some future plans to also quantize to GPTQ/AWQ
fzyzcjy changed discussion title from Supporting GPTQ to Supporting GPTQ/AWQ