Cyclone300/WolfGang300

Dolphin3.0 Llama3.2 1B — GGUF (mobile-optimized)

GGUF quants of dphn/Dolphin3.0-Llama3.2-1B, tuned for on-device / phone inference. Quantized with the i1 importance matrix (imatrix) for better quality at low bit widths.

Files

File	Size	Quant	Notes
Dolphin3.0-Llama3.2-1B.i1-IQ1_S.gguf	394 MB	IQ1_S	Smallest. Lowest memory, lowest quality.
Dolphin3.0-Llama3.2-1B.i1-Q4_K_M.gguf	808 MB	Q4_K_M	Recommended. Best coherence per byte.

Phone usage

Requires a llama.cpp build with a GGUF loader (MLC, Ollama, llama.cpp, ChatterUI, PocketPal, MLC Chat, or an iOS/Android app with llama.cpp support).

llama.cpp CLI:

llama-cli -m Dolphin3.0-Llama3.2-1B.i1-Q4_K_M.gguf \
  -ngl 99 -c 4096 -n 512 \
  --jinja \
  -p "<|im_start|>user
Hi<|im_end|>
<|im_start|>assistant
"

Chat template (ChatML, as used by Dolphin):

<|im_start|>system
You are a helpful assistant.<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant

Recommended phone settings:

• -c 4096 context (1B degrades past ~4k; raise only if you have RAM headroom)
• -ngl 99 offload everything to GPU/NPU
• Prefer Q4_K_M. Drop to IQ1_S only on very low-RAM devices.

Provenance

Quants mirrored from mradermacher/Dolphin3.0-Llama3.2-1B-i1-GGUF. Base model by the Dolphin project (dphn). No weights were modified in this mirror.