huihui-ai/Huihui-Qwen3.5-9B-abliterated
Image-Text-to-Text • 10B • Updated • 7.88k • • 171
Low-VRAM Models in Testing
Whether or not a model can run with 6GB of VRAM or less. Models are being tested against conversational interactions and instruction following.
Note Q4_K_M results are not promising thus far. Time to first token is ~50 seconds on cold start and ~30 seconds thereafter. Might be fine for some things, but quantizing it further may erode any of the gains from the extra 1B in training data vs using the Qwen3-8B version. Still testing.
-
huihui-ai/Huihui-Qwen3.5-4B-abliterated
Image-Text-to-Text • 5B • Updated • 19.8k • • 56 -
openbmb/MiniCPM5-2B
Text Generation • 3B • Updated • 1.07M • 1.84k
unsloth/gemma-4-12B-it-qat-GGUF
Any-to-Any • 12B • Updated • 208k • 609Note Too slow to be useful at 12B. The huihui-ai abliterated version E4B version is better in every regard. Don't waste your time with this one.
-
Qwen/Qwen3-4B-Instruct-2507
Text Generation • 4B • Updated • 2.6M • • 1.05k -
huihui-ai/Huihui-Qwen3-4B-Instruct-2507-abliterated
Text Generation • 4B • Updated • 5.32k • • 45 -
unsloth/gemma-4-E4B-it-qat-GGUF
Any-to-Any • 7B • Updated • 125k • 231