What is the difference q4_0 / q4_2 / q4_3 ???
#5
by
vanSamstroem
- opened
Thank you very much but what is the difference between these *.bin files?
alpaca-lora-65B.GGML.q4_0.bin
alpaca-lora-65B.GGML.q4_2.bin
alpaca-lora-65B.GGML.q4_3.bin
Downloaded this one "alpaca-lora-65B.GGML.q4_3.bin" because I thought it was the newest one but I can't get it to work with "Alpaca Electron App" nor via Terminal with llama.cpp on M1 Max 64GB Macbook. Failed to load model (bad f16 value 6)...
Sorry for the lack of explanation. I've now updated the README to explain.
All of the files will require recent versions of llama.cpp to run. So it's likely your app isn't using a recent enough version of the llama.cpp code.
TheBloke
changed discussion status to
closed
Thank you very much!