Interesting model with the right direction
#1
by thermi6 - opened
Thank you for the model. I think it is going into the right direction. Will you put some more work into this to bring it up to par with the original, but make it less chatty in its thinking?
I may put more work into it in the future but currently I don’t have much compute / GPU credits left
I wanted to use this model (Nanbeige) and I had the exact same complaint I did with the older ones <= 4.1.
A lot of people don't know about these smaller releases and stick with the brand names, glad you found that one.
It just burns through tokens. Not only did you fix the problem, but then you quantized it, with QAT.
Respect.