ggml_alpaca_30b_q4 ?
#5
by DaveScream - opened
can you please make quantized 4bit version of llama30b + alpaca lora 30b?
sharing alpaca_lora 30b: https://github.com/tloen/alpaca-lora/issues/68
https://github.com/johnsmith0031/alpaca_lora_4bit
the 4bit llama 30b appears to be here, have you tried using it with alpaca lora?