Q4_K_M is Very Unstable
#1
by
Koitenshin
- opened
I tested the Q4_K_M quant and the model just collapses after a single message, but some times you'll get lucky and be able to get 3 out of it.
I tested the Q4_K_M quant and the model just collapses after a single message, but some times you'll get lucky and be able to get 3 out of it.