| --- |
| library_name: gguf |
| tags: |
| - gguf |
| - llama |
| - dolphin |
| - llama-3.2 |
| - 1b |
| - mobile |
| - on-device |
| base_model: dphn/Dolphin3.0-Llama3.2-1B |
| license: apache-2.0 |
| --- |
| |
| # Dolphin3.0 Llama3.2 1B — GGUF (mobile-optimized) |
|
|
| GGUF quants of [dphn/Dolphin3.0-Llama3.2-1B](https://huggingface.co/dphn/Dolphin3.0-Llama3.2-1B), |
| tuned for on-device / phone inference. Quantized with the `i1` importance matrix |
| (imatrix) for better quality at low bit widths. |
|
|
| ## Files |
|
|
| | File | Size | Quant | Notes | |
| |---|---|---|---| |
| | `Dolphin3.0-Llama3.2-1B.i1-IQ1_S.gguf` | 394 MB | IQ1_S | Smallest. Lowest memory, lowest quality. | |
| | `Dolphin3.0-Llama3.2-1B.i1-Q4_K_M.gguf` | 808 MB | Q4_K_M | **Recommended.** Best coherence per byte. | |
| |
| ## Phone usage |
| |
| Requires a [llama.cpp](https://github.com/ggml-org/llama.cpp) build with a GGUF loader |
| (MLC, Ollama, llama.cpp, ChatterUI, PocketPal, MLC Chat, or an iOS/Android app with |
| llama.cpp support). |
| |
| llama.cpp CLI: |
| |
| ```bash |
| llama-cli -m Dolphin3.0-Llama3.2-1B.i1-Q4_K_M.gguf \ |
| -ngl 99 -c 4096 -n 512 \ |
| --jinja \ |
| -p "<|im_start|>user |
| Hi<|im_end|> |
| <|im_start|>assistant |
| " |
| ``` |
| |
| Chat template (ChatML, as used by Dolphin): |
| |
| ``` |
| <|im_start|>system |
| You are a helpful assistant.<|im_end|> |
| <|im_start|>user |
| {prompt}<|im_end|> |
| <|im_start|>assistant |
| ``` |
| |
| Recommended phone settings: |
| - `-c 4096` context (1B degrades past ~4k; raise only if you have RAM headroom) |
| - `-ngl 99` offload everything to GPU/NPU |
| - Prefer **Q4_K_M**. Drop to IQ1_S only on very low-RAM devices. |
|
|
| ## Provenance |
|
|
| Quants mirrored from [`mradermacher/Dolphin3.0-Llama3.2-1B-i1-GGUF`](https://huggingface.co/mradermacher/Dolphin3.0-Llama3.2-1B-i1-GGUF). |
| Base model by the Dolphin project (`dphn`). No weights were modified in this mirror. |
|
|
