Thank you for your Quant!
DOwnloading it as we speak. I will report back. Thank you :)
i have noticed that while running your qunats i do not see mistral error message while starting up the server which i see in the cyanwiki's quants. My OCD prefers your quants lol
Amazing quant! Runs perfectly on 2x6000 pros! Life is good.
Thanks for the quant - any chance you could do a 5bit one as well? Thank you
Thanks for the quant - any chance you could do a 5bit one as well? Thank you
Maybe what you meant was GPTQ mixed? We don’t have plans for that at the moment.
DOwnloading it as we speak. I will report back. Thank you :)
i have noticed that while running your qunats i do not see mistral error message while starting up the server which i see in the cyanwiki's quants. My OCD prefers your quants lol
For some reason, cyankiwi quant is faster for me - I get 40 t/s with that one, and 27 t/s with QuantTrio on dual DGX Spark cluster....
I'll run some benchmarks comparing the quants later today
DOwnloading it as we speak. I will report back. Thank you :)
i have noticed that while running your qunats i do not see mistral error message while starting up the server which i see in the cyanwiki's quants. My OCD prefers your quants lol
For some reason, cyankiwi quant is faster for me - I get 40 t/s with that one, and 27 t/s with QuantTrio on dual DGX Spark cluster....
it is opposite for me on 2X6000 Pros.