Is it work to quantize an existing LLM into 1.58bit?
#1
by
SilverJim
- opened
Hello, I wonder Is it work to quantize an existing LLM into 1.58bit?
About 1.58b quantization, I think it loses so much precision to quantize an existing LLM into 1.58bit, it is seems 1.58bit quantization can only get a good result when the LLM is trained as a 1.58bit model from beginning.