QUANTIZED MODEL
#1
by parthwagh - opened
team mradermacher
Could you please provide quantized version for this model?
I have RTX 4090.
Thank you so much for open sourcing this model.
We will never forget your efforts.
Thanks.
I don't think it can be fit into 1 4090 even if 4bit quant. :(
It will, it is lower than 70B models