| --- |
| license: apache-2.0 |
| tags: |
| - gguf |
| - llama.cpp |
| - onboard |
| - understand |
| --- |
| |
| # OnBoard recommended models (GGUF) |
|
|
| The models OnBoard's installer offers, mirrored here for |
| stable hosting. All files are 4-bit GGUF quantizations for |
| llama.cpp-based serving (shipped with OnBoard/Understand as `ullama`), all |
| under the Apache-2.0 license, mirrored unmodified from the source repos below. |
|
|
| Every model here has been through our own qualification testing on real |
| Understand project data: a chat test that measures how well the model answers |
| questions about a codebase using OnBoard's analysis tools, and a code-summary |
| benchmark that grades generated overviews for accuracy against the source. |
| The notes below come from those measurements, not from vendor claims. That |
| said, AI is AI — every model gets things wrong, so treat any answer as a |
| starting point and take it with a grain of salt. |
|
|
| ## Which model should I use? |
|
|
| Run `ullama-recommend` (shipped with OnBoard and Understand) — it looks at |
| your machine's memory and suggests the right size. In short: pick the largest |
| one your machine runs comfortably, and prefer the gemma models when you want |
| the best chat answers. |
|
|
| | Model | Download | Why you would pick it | |
| |---|---|---| |
| | gpt-oss-120b | 63 GB (2 parts) | The most accurate code summaries in our testing, and quick for its size. Needs roughly a 128 GB machine. | |
| | Qwen3.6-35B-A3B | 22.4 GB | Highly accurate code summaries, and much faster than other models this large. | |
| | Gemma 4 12B | 7.1 GB | The best chat answers of any model we tested. | |
| | Qwen3.5 9B | 5.7 GB | Did well in chat testing, and digs a little deeper into code before answering. | |
| | Gemma 4 E4B | 5.0 GB | Did well in our chat testing. | |
| | Gemma 4 E2B | 3.1 GB | OnBoard's default model. Good at both chat and code summaries. | |
| | Qwen3.5 2B | 1.3 GB | Writes good code summaries remarkably fast, but struggled in our chat testing. | |
|
|
|
|
|
|
| ## Files, licensing, and provenance |
|
|
| | File | Model | Company (Country) | License | Source | |
| |---|---|---|---|---| |
| | gpt-oss-120b-Q4_K_M-0000?-of-00002.gguf | gpt-oss-120b | OpenAI (United States) | Apache-2.0 | [unsloth/gpt-oss-120b-GGUF](https://huggingface.co/unsloth/gpt-oss-120b-GGUF) | |
| | Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf | Qwen3.6-35B-A3B | Alibaba (China) | Apache-2.0 | [unsloth/Qwen3.6-35B-A3B-GGUF](https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF) | |
| | gemma-4-12b-it-Q4_K_M.gguf | Gemma 4 12B | Google (United States) | Apache-2.0 | [unsloth/gemma-4-12b-it-GGUF](https://huggingface.co/unsloth/gemma-4-12b-it-GGUF) | |
| | Qwen3.5-9B-Q4_K_M.gguf | Qwen3.5 9B | Alibaba (China) | Apache-2.0 | [unsloth/Qwen3.5-9B-GGUF](https://huggingface.co/unsloth/Qwen3.5-9B-GGUF) | |
| | gemma-4-E4B-it-Q4_K_M.gguf | Gemma 4 E4B | Google (United States) | Apache-2.0 | [unsloth/gemma-4-E4B-it-GGUF](https://huggingface.co/unsloth/gemma-4-E4B-it-GGUF) | |
| | gemma-4-E2B-it-Q4_K_M.gguf | Gemma 4 E2B | Google (United States) | Apache-2.0 | [unsloth/gemma-4-E2B-it-GGUF](https://huggingface.co/unsloth/gemma-4-E2B-it-GGUF) | |
| | Qwen3.5-2B-Q4_K_M.gguf | Qwen3.5 2B | Alibaba (China) | Apache-2.0 | [unsloth/Qwen3.5-2B-GGUF](https://huggingface.co/unsloth/Qwen3.5-2B-GGUF) | |
|
|
| Quantization policy: Q4_K_M for models ≤12B; Unsloth Dynamic (UD-Q4_K_XL) for |
| the large MoE, where the dynamic quant's quality edge matters most. gpt-oss-120b |
| is a two-part split: download both parts into the same directory and point the |
| server at part 1 — llama.cpp finds the second part automatically. |
|
|
| All models retain their original Apache-2.0 licenses; no modifications were |
| made beyond mirroring the original GGUF files. |
|
|
