(Un)officially support 1x96GB?
#4
by JoeSmith245 - opened
Can't wait to try this, thanks.
Worth noting: M2.5 actually runs pretty well at UD-IQ2_M, in 96GB. A bit limited in context, but it works. That brings it within reach of non-commercial enthusiasts with 4x3090, or 3x32GB.
I notice that your recommended configs are for 4x96GB and 8x96GB, so you have the right hardware, and could "unofficially support" 1x96GB too :)
This is really just me begging you to keep (at least the lower end of) your models at 200-220B, even if you add multimodality in future. It's a great level of complexity, and just within reach for a lot of us, but I suppose it could help you get more attention too, if you unofficially support systems that more people have.
JoeSmith245 changed discussion title from Officially support 1x96GB? to (Un)officially support 1x96GB?
EDIT: never mind, the license is a bad joke.