leon-se/gemma-3-27b-it-FP8-Dynamic

27B • Updated Apr 8, 2025 • 663 • 20

Note Runs well on vLLM, will not fit standard sglang. Very good model for the size.

dddsaty/phi-4-GPTQ-8bit

Text Generation • 15B • Updated Jan 11, 2025 • 5 • 1

Note Very fast with sglang, can improve even more with draft easiest-ai-shawn/Phi-4-EAGLE3-sharegpt-unfiltered

cyankiwi/Qwen3-Coder-30B-A3B-Instruct-AWQ-4bit

Text Generation • 5B • Updated Mar 23 • 54.5k • 49

Note Very good at tool call and instruction following, prone to unexpected hallucinations.

gaunernst/gemma-3-27b-it-int4-awq

Image-Text-to-Text • 6B • Updated Apr 6, 2025 • 26.1k • 40

Note Good balance of size and ease of running

google/gemma-3-12b-it

Image-Text-to-Text • Updated Mar 21, 2025 • 2.6M • • 710

Note On the smaller side, I recommend larger models