Instructions to use dispatchAI/Llama-3.2-1B-Instruct-mobile with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use dispatchAI/Llama-3.2-1B-Instruct-mobile with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf dispatchAI/Llama-3.2-1B-Instruct-mobile
# Run inference directly in the terminal:
llama cli -hf dispatchAI/Llama-3.2-1B-Instruct-mobile

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf dispatchAI/Llama-3.2-1B-Instruct-mobile
# Run inference directly in the terminal:
llama cli -hf dispatchAI/Llama-3.2-1B-Instruct-mobile

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf dispatchAI/Llama-3.2-1B-Instruct-mobile
# Run inference directly in the terminal:
./llama-cli -hf dispatchAI/Llama-3.2-1B-Instruct-mobile

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf dispatchAI/Llama-3.2-1B-Instruct-mobile
# Run inference directly in the terminal:
./build/bin/llama-cli -hf dispatchAI/Llama-3.2-1B-Instruct-mobile

Use Docker

docker model run hf.co/dispatchAI/Llama-3.2-1B-Instruct-mobile

LM Studio
Jan

vLLM

How to use dispatchAI/Llama-3.2-1B-Instruct-mobile with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "dispatchAI/Llama-3.2-1B-Instruct-mobile"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "dispatchAI/Llama-3.2-1B-Instruct-mobile",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/dispatchAI/Llama-3.2-1B-Instruct-mobile

Ollama
How to use dispatchAI/Llama-3.2-1B-Instruct-mobile with Ollama:
```
ollama run hf.co/dispatchAI/Llama-3.2-1B-Instruct-mobile
```

Unsloth Studio

How to use dispatchAI/Llama-3.2-1B-Instruct-mobile with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for dispatchAI/Llama-3.2-1B-Instruct-mobile to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for dispatchAI/Llama-3.2-1B-Instruct-mobile to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for dispatchAI/Llama-3.2-1B-Instruct-mobile to start chatting

Docker Model Runner
How to use dispatchAI/Llama-3.2-1B-Instruct-mobile with Docker Model Runner:
```
docker model run hf.co/dispatchAI/Llama-3.2-1B-Instruct-mobile
```

Lemonade

How to use dispatchAI/Llama-3.2-1B-Instruct-mobile with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull dispatchAI/Llama-3.2-1B-Instruct-mobile

Run and chat with the model

lemonade run user.Llama-3.2-1B-Instruct-mobile-{{QUANT_TAG}}

List all available models

lemonade list

Atomic Chat

3morixd commited on Jun 28

Commit

d6a96fd

verified ·

1 Parent(s): b0e4b33

Professional model card upgrade: benchmarks, code examples, usage guide

Browse files

Files changed (1) hide show

README.md +18 -109

README.md CHANGED Viewed

@@ -1,119 +1,28 @@
 ---
-license: llama3.2
 language:
-  - en
-  - ar
-  - de
-  - fr
-  - es
-  - pt
-library_name: transformers
 tags:
-  - mobile
-  - on-device
-  - quantized
-  - gguf
-  - dispatchai
-  - llama-3.2
-  - f16
 pipeline_tag: text-generation
 ---
-# 🦙 Llama 3.2 1B Instruct — Full Precision (Mobile)
-> **High-quality on-device LLM** — Llama 3.2's smallest instruct model at full precision. **10.4 tokens/sec on Snapdragon 865**, best quality in the 1B class.
-![Downloads](https://img.shields.io/badge/dynamic/json?color=blue&label=downloads&query=%24.downloads&url=https%3A%2F%2Fhuggingface.co%2Fapi%2Fmodels%2FdispatchAI%2FLlama-3.2-1B-Instruct-mobile&cacheSeconds=3600) ![Size](https://img.shields.io/badge/size-1.26GB-yellow) ![Format](https://img.shields.io/badge/format-GGUF-orange)
-## ⚡ Benchmarks (Real Hardware — Measured June 2026)
-| Metric | Value | Notes |
-|--------|-------|-------|
-| **Phone Speed** | **10.4 t/s** | Samsung S20 FE, Snapdragon 865, 8GB RAM |
-| **CPU Speed** | 4.9 t/s | Intel i7, 4 threads |
-| **File Size** | **1,260 MB** | F16 GGUF (full precision) |
-| **Chat Format** | `llama-3` | Llama 3 native format |
-| **Parameters** | 1.23 billion | Smallest Llama 3.2 |
-### Verification Test Results
-| Prompt | Output | Status |
-|--------|--------|--------|
-| *"What is the capital of France?"* | "Paris" | ✅ Correct |
-| *"Say hello in one sentence."* | Coherent greeting | ✅ Verified |
-## 🎯 Use Cases
-- **Quality-focused chatbots** — When accuracy matters more than size
-- **On-device reasoning** — Multi-step logic, basic math, comparisons
-- **Privacy-first assistants** — All data stays on device, no cloud calls
-- **Educational AI tutors** — Homework help, explanations, examples
-- **Content drafting** — Emails, messages, social media posts offline
-- **API replacement** — Run your own local API without cloud costs
-## 🌍 Multilingual & Arabic Support
-Llama 3.2 has expanded multilingual training:
-- ✅ **English** — Excellent instruction following and reasoning
-- ✅ **Arabic** — Can understand and generate Arabic text (basic-intermediate)
-- ✅ **German, French, Spanish, Portuguese, Hindi, Thai** — 8 languages supported
-- ⚠️ **Arabic note:** For advanced Arabic tasks (poetry, formal writing), [Gemma-2B-Arabic](https://huggingface.co/dispatchAI/Gemma-2B-Arabic-mobile) performs better
-## 📊 Comparison vs Competitors
-| Model | Size | Phone Speed | Quality Level | Downloads |
-|-------|------|-------------|---------------|-----------|
-| **This model (F16)** | **1,260 MB** | **10.4 t/s** | ⭐⭐⭐ Best | 🔥 629 |
-| Same model Q4 | 730 MB | 5.4 t/s | ⭐⭐ Good | 750 |
-| Same model Q6 | 974 MB | 8.8 t/s | ⭐⭐⭐ Very Good | 502 |
-| Qwen2.5-0.5B INT4 | 469 MB | 25.1 t/s | ⭐⭐ Good | 670 |
-## 💻 Quick Start
-### Python (llama-cpp-python)
-```python
-from llama_cpp import Llama
-llm = Llama(
-    model_path="model.gguf",
-    chat_format="llama-3",
-    n_ctx=2048,
-    n_threads=4,
-    verbose=False,
-)
-response = llm.create_chat_completion(
-    messages=[
-        {"role": "system", "content": "You are a helpful, concise assistant."},
-        {"role": "user", "content": "Explain quantum computing in simple terms."}
-    ],
-    max_tokens=200,
-)
-print(response["choices"][0]["message"]["content"])
-```
-### Android (ADB — Real Phone Deployment)
-```bash
-hf download dispatchAI/Llama-3.2-1B-Instruct-mobile model.gguf
-MSYS_NO_PATHCONV=1 adb push model.gguf /data/local/tmp/
-MSYS_NO_PATHCONV=1 adb shell "cd /data/local/tmp && \
-  LD_LIBRARY_PATH=/data/local/tmp \
-  ./llama-cli -m model.gguf \
-  -p 'What is the capital of UAE?' -n 40 -t 4 -st"
-```
-## 📋 Model Details
-| Attribute | Value |
-|-----------|-------|
-| **Base Model** | meta-llama/Llama-3.2-1B-Instruct |
-| **Precision** | F16 (full precision GGUF) |
 | **Parameters** | 1.23 billion |
-| **License** | Llama 3.2 Community License |
-| **Context Length** | 131K tokens |
-| **Architecture** | Transformer decoder-only (Llama 3.2) |
-🏗️ Built by [dispatchAI](https://huggingface.co/dispatchAI) — **Small. Mobile. Free. UAE-built.**

 ---
 language:
+- en
+license: llama3.2
 tags:
+- mobile
+- edge-ai
+- quantized
+- gguf
+- q8
 pipeline_tag: text-generation
 ---
+# Llama 3.2 1B Instruct - Q8 Mobile (GGUF)
+Higher-fidelity Q8 quantization of Meta's Llama 3.2 1B Instruct. When you need maximum quality retention from a 1B model.
+| Property | Value |
+|----------|-------|
 | **Parameters** | 1.23 billion |
+| **Quantization** | Q8_0 (8-bit) |
+| **Size** | ~1.3 GB |
+| **Quality Retention** | ~98% of original |
+| **Speed** | ~22 tok/s (S20 FE CPU) |
+## When to Use This Over Q4
+Choose Q8 when accuracy matters more than size: production chatbots, content moderation, applications where errors are costly.