Implementation experiences.
With 13,000 downloads as of this writing, have any of you run the model successully on midrange hardware?
How do your results differ with varying quants?
One thing I'll say is it REALLY tends to overthink, like simple queries will result in like a thousand tokens
BUT it doesn't get stuck in any weird loops, it does figure everything out by the end, and gives a perfect final answer, so... At least there's that..?
Thanks bartowski. I can't run it yet but I'll share observations when I can.
In 1991 if there were 100 people downloading frontier work like this, there'd be 90 people talking about it on usenet.
I downloaded IQ2_S on my desktop but cannot load it in LMStudio. I updated to the latest beta version but still no dice.
I get the following error:
🥲 Failed to load the model
Failed to load model
error loading model: missing tensor 'blk.92.nextn.embed_tokens.weight'