README.md

metadata

library_name: gguf
tags:
  - gguf
  - llama
  - dolphin
  - llama-3.2
  - 1b
  - mobile
  - on-device
base_model: dphn/Dolphin3.0-Llama3.2-1B
license: apache-2.0

Dolphin3.0 Llama3.2 1B — GGUF (mobile-optimized)

GGUF quants of dphn/Dolphin3.0-Llama3.2-1B, tuned for on-device / phone inference. Quantized with the i1 importance matrix (imatrix) for better quality at low bit widths.

Files

File	Size	Quant	Notes
Dolphin3.0-Llama3.2-1B.i1-IQ1_S.gguf	394 MB	IQ1_S	Smallest. Lowest memory, lowest quality.
Dolphin3.0-Llama3.2-1B.i1-Q4_K_M.gguf	808 MB	Q4_K_M	Recommended. Best coherence per byte.

Phone usage

Requires a llama.cpp build with a GGUF loader (MLC, Ollama, llama.cpp, ChatterUI, PocketPal, MLC Chat, or an iOS/Android app with llama.cpp support).

llama.cpp CLI:

llama-cli -m Dolphin3.0-Llama3.2-1B.i1-Q4_K_M.gguf \
  -ngl 99 -c 4096 -n 512 \
  --jinja \
  -p "<|im_start|>user
Hi<|im_end|>
<|im_start|>assistant
"

Chat template (ChatML, as used by Dolphin):

<|im_start|>system
You are a helpful assistant.<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant

Recommended phone settings:

• -c 4096 context (1B degrades past ~4k; raise only if you have RAM headroom)
• -ngl 99 offload everything to GPU/NPU
• Prefer Q4_K_M. Drop to IQ1_S only on very low-RAM devices.

Provenance

Quants mirrored from mradermacher/Dolphin3.0-Llama3.2-1B-i1-GGUF. Base model by the Dolphin project (dphn). No weights were modified in this mirror.