can't use
#1
by gooes - opened
i can't use this models even start it up.then ,when i started,it take some bad outputs.
I had similar problems with an AWQ quant, but never with this one. The only bad output I had was when I attempted to quantize the KV-cache. How are you calling the model? Are you getting any errors or other diagnostic messages?