barozp/Qwen3.6-28B-REAP20-A3B-GGUF

Text Generation • 28B • Updated Apr 19 • 12.2k • 67

Note - gpu+cpu 36 t/s on 8G, works ok on 4G GPU too but 15 t/s - good at coding - understands the tools llama-server.exe -m "models/Qwen3.6-28B-REAP20-A3B-Q4_K_M.gguf" --n-cpu-moe 35 --fit on --ctx-size 40960 llama-server -hf barozp/Qwen3.6-28B-REAP20-A3B-GGUF:Q4_K_M --ctx-size 30720

unsloth/GLM-4.7-Flash-REAP-23B-A3B-GGUF

Text Generation • 23B • Updated Jan 28 • 20.9k • 264

Note - runs on gpu and cpu but reap makes it 30 t/s - not bad at coding but Quen3.6 is better .\llama-b9631-bin-win-cuda-13.3-x64\llama-server.exe -m "models/GLM-4.7-Flash-REAP-23B-A3B-Q4_K_M.gguf" --n-cpu-moe 35 --cache-type-k q8_0 --cache-type-v q4_0 --fit on --temp 1.0 --top-p 0.95 --min-p 0.01 --ctx-size 40960

llmfan46/gemma-4-E4B-it-ultra-uncensored-heretic-GGUF

Any-to-Any • 8B • Updated May 28 • 54.4k • 166

Note - fits fully in 8G generating 70 t/s - problems with edit tool - needs very clear instructions - absolutly uncensored llama-server.exe -m "models\gemma-4-E4B-it-ultra-uncensored-heretic-Q5_K_M.gguf" --ctx-size 131072 --no-mmap --mlock --temp 0.6 --top-p 0.95 --top-k 64

google/gemma-4-E4B-it-qat-q4_0-gguf

Any-to-Any • 7B • Updated 21 days ago • 149k • 112

Note - fast and fits in 8GB VRAM - has problem with edit tool but works around it .\llama-b9631-bin-win-cuda-13.3-x64\llama-server.exe -m "models/gemma-4-E4B_q4_0-it.gguf" --ctx-size 131072 --no-mmap --mlock

unsloth/gemma-4-E4B-it-qat-GGUF

Any-to-Any • 7B • Updated 22 days ago • 289k • 151

Note - fits in 8GB todo: --spec-type draft-mtp --spec-draft-n-max 4 -fit off llama-server.exe -m "models/gemma-4-E4B-it-qat-UD-Q4_K_XL.gguf" --ctx-size 102400 --no-mmap

unsloth/gemma-4-E2B-it-qat-GGUF

Any-to-Any • 5B • Updated 22 days ago • 124k • 69

Note - works on 4G with over 100 t/s - as smart as 5B can be, big context, basic coding, can read images too. llama-server -hf unsloth/gemma-4-E2B-it-qat-GGUF:UD-Q4_K_XL -fa on -np 1 --spec-type draft-mtp --spec-draft-n-max 4 -fit off --ctx-size 90112 --mlock --no-mmap -ngl all

Jackrong/Qwopus3.5-4B-Coder-GGUF

Image-Text-to-Text • 4B • Updated May 30 • 6.72k • 30

Note - works on 4G but slow (15 t/s), needs tuning - its a real coder, started to read the pi source code to undersand tool calls better :D llama-server -hf Jackrong/Qwopus3.5-4B-Coder-GGUF:Q4_K_M --spec-type draft-mtp --spec-draft-n-max 2 --no-mmproj-offload --ctx-size 131072 --rope-scaling yarn --rope-scale 4 --yarn-orig-ctx 32768 --no-mmap --mlock

unsloth/Qwen3.5-4B-MTP-GGUF

Image-Text-to-Text • 4B • Updated May 16 • 73k • 71

Note - fits fully in 4G - understands tooling todo: tuning, its very slow llama-server -hf unsloth/Qwen3.5-4B-MTP-GGUF:UD-Q4_K_XL -fa on -np 1 --spec-type draft-mtp --spec-draft-n-max 6 --ctx-size 15000 --mlock