2.4-2.5 bpw request?
#1
by
Michund10s9
- opened
Heya, any chance we can get a 2.4/2.5 quant? That places the model just beyond the massive inflection point of KL divergence (at least for devstral https://huggingface.co/turboderp/Devstral-2-123B-Instruct-2512-exl3)
and also around the perfect amount for 80GB VRAM
I'm going to do some optimized versions shortly.