File size: 3,617 Bytes
556e392 1c707f2 a46248e 556e392 1c707f2 556e392 a46248e 556e392 a46248e 556e392 a46248e 556e392 a46248e eb9ae04 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 | ---
license: apache-2.0
tags:
- gguf
- llama.cpp
- onboard
- understand
---
# OnBoard recommended models (GGUF)
The models OnBoard's installer offers, mirrored here for
stable hosting. All files are 4-bit GGUF quantizations for
llama.cpp-based serving (shipped with OnBoard/Understand as `ullama`), all
under the Apache-2.0 license, mirrored unmodified from the source repos below.
Every model here has been through our own qualification testing on real
Understand project data: a chat test that measures how well the model answers
questions about a codebase using OnBoard's analysis tools, and a code-summary
benchmark that grades generated overviews for accuracy against the source.
The notes below come from those measurements, not from vendor claims. That
said, AI is AI — every model gets things wrong, so treat any answer as a
starting point and take it with a grain of salt.
## Which model should I use?
Run `ullama-recommend` (shipped with OnBoard and Understand) — it looks at
your machine's memory and suggests the right size. In short: pick the largest
one your machine runs comfortably, and prefer the gemma models when you want
the best chat answers.
| Model | Download | Why you would pick it |
|---|---|---|
| gpt-oss-120b | 63 GB (2 parts) | The most accurate code summaries in our testing, and quick for its size. Needs roughly a 128 GB machine. |
| Qwen3.6-35B-A3B | 22.4 GB | Highly accurate code summaries, and much faster than other models this large. |
| Gemma 4 12B | 7.1 GB | The best chat answers of any model we tested. |
| Qwen3.5 9B | 5.7 GB | Did well in chat testing, and digs a little deeper into code before answering. |
| Gemma 4 E4B | 5.0 GB | Did well in our chat testing. |
| Gemma 4 E2B | 3.1 GB | OnBoard's default model. Good at both chat and code summaries. |
| Qwen3.5 2B | 1.3 GB | Writes good code summaries remarkably fast, but struggled in our chat testing. |
## Files, licensing, and provenance
| File | Model | Company (Country) | License | Source |
|---|---|---|---|---|
| gpt-oss-120b-Q4_K_M-0000?-of-00002.gguf | gpt-oss-120b | OpenAI (United States) | Apache-2.0 | [unsloth/gpt-oss-120b-GGUF](https://huggingface.co/unsloth/gpt-oss-120b-GGUF) |
| Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf | Qwen3.6-35B-A3B | Alibaba (China) | Apache-2.0 | [unsloth/Qwen3.6-35B-A3B-GGUF](https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF) |
| gemma-4-12b-it-Q4_K_M.gguf | Gemma 4 12B | Google (United States) | Apache-2.0 | [unsloth/gemma-4-12b-it-GGUF](https://huggingface.co/unsloth/gemma-4-12b-it-GGUF) |
| Qwen3.5-9B-Q4_K_M.gguf | Qwen3.5 9B | Alibaba (China) | Apache-2.0 | [unsloth/Qwen3.5-9B-GGUF](https://huggingface.co/unsloth/Qwen3.5-9B-GGUF) |
| gemma-4-E4B-it-Q4_K_M.gguf | Gemma 4 E4B | Google (United States) | Apache-2.0 | [unsloth/gemma-4-E4B-it-GGUF](https://huggingface.co/unsloth/gemma-4-E4B-it-GGUF) |
| gemma-4-E2B-it-Q4_K_M.gguf | Gemma 4 E2B | Google (United States) | Apache-2.0 | [unsloth/gemma-4-E2B-it-GGUF](https://huggingface.co/unsloth/gemma-4-E2B-it-GGUF) |
| Qwen3.5-2B-Q4_K_M.gguf | Qwen3.5 2B | Alibaba (China) | Apache-2.0 | [unsloth/Qwen3.5-2B-GGUF](https://huggingface.co/unsloth/Qwen3.5-2B-GGUF) |
Quantization policy: Q4_K_M for models ≤12B; Unsloth Dynamic (UD-Q4_K_XL) for
the large MoE, where the dynamic quant's quality edge matters most. gpt-oss-120b
is a two-part split: download both parts into the same directory and point the
server at part 1 — llama.cpp finds the second part automatically.
All models retain their original Apache-2.0 licenses; no modifications were
made beyond mirroring the original GGUF files.
|