---
license: apache-2.0
tags:
- pocketllm
- on-device
- edge-ai
- mobile
- android
- offline
- mediapipe
- gguf
- llama
- gemma
- qwen
- phi
language:
- en
- hi
- multilingual
pipeline_tag: text-generation
---
# PocketLLM Model Collection
**The official model collection for [PocketLLM](https://github.com/amareshhebbar/PocketLLM)**
*Private • Offline • On-Device AI for Android*
[](https://github.com/amareshhebbar/PocketLLM)
[](https://github.com/amareshhebbar/PocketLLM)
[](LICENSE)
---
## What is PocketLLM?
PocketLLM is an Android app that runs large language models **completely on your device** — no internet required during inference, no data sent to the cloud, no subscriptions.
This repository hosts the curated model collection optimized for mobile edge inference using:
- **MediaPipe LLM Inference** (for `.bin` and `.task` files — Gemma family)
- **llama.cpp via llama.rn** (for `.gguf` files — Llama, Phi, Qwen, SmolLM, Gemma 3)
---
## Model Catalog
### ⚡ Ultra Fast — Under 1 GB, fits any Android phone
| Model | File | Size | Format | RAM |
|-------|------|------|--------|-----|
| Qwen 2.5 0.5B | `qwen2.5-0.5b-instruct-q4_k_m.gguf` | 0.4 GB | GGUF | 2 GB |
| Gemma 3 1B | `gemma-3-1b-it-q4_k_m.gguf` | 0.7 GB | GGUF | 2 GB |
| Llama 3.2 1B | `llama-3.2-1b-instruct-q4_k_m.gguf` | 0.8 GB | GGUF | 3 GB |
| SmolLM2 1.7B | `smollm2-1.7b-instruct-q4_k_m.gguf` | 1.0 GB | GGUF | 3 GB |
### ⚖️ Balanced — 1–2 GB, great quality on mid-range phones
| Model | File | Size | Format | RAM |
|-------|------|------|--------|-----|
| Gemma 1.1 2B (CPU) | `gemma-1.1-2b-it-cpu-int4.bin` | 1.35 GB | MediaPipe | 4 GB |
| Gemma 1.1 2B (GPU) | `gemma-1.1-2b-it-gpu-int4.bin` | 1.35 GB | MediaPipe | 4 GB |
| Llama 3.2 3B | `llama-3.2-3b-instruct-q4_k_m.gguf` | 2.0 GB | GGUF | 4 GB |
### 🚀 Powerful — Best quality, needs 5GB+ RAM
| Model | File | Size | Format | RAM |
|-------|------|------|--------|-----|
| Phi-3.5 Mini | `phi-3.5-mini-instruct-q4_k_m.gguf` | 2.4 GB | GGUF | 5 GB |
| Gemma 3 4B | `gemma-3-4b-it-q4_k_m.gguf` | 2.8 GB | GGUF | 6 GB |
---
## How to Use in PocketLLM
Models are downloaded directly inside the PocketLLM app. Open the **Model Store** tab, select a model, and tap Download. The app handles everything automatically.
**Direct download URLs:**
```
https://huggingface.co/AmareshHebbar/pocketllm-models/resolve/main/
```
---
## Fine-Tuning Roadmap
> *Coming soon — within the next 2 weeks*
We are fine-tuning these base models specifically for **on-device conversational AI** on mobile:
### Goals
- Better persona adherence (the model stays in character consistently)
- Shorter, more natural responses (base models tend to be verbose on mobile)
- Improved memory utilization (uses injected memories naturally)
- Better instruction following for small context windows (2048 tokens)
- Hindi + English code-switching support (for India market)
### Training approach
- **Base:** Llama 3.2 1B, Gemma 3 1B (smallest models first — fastest iteration)
- **Method:** QLoRA fine-tuning (4-bit, similar to the AxioMapper training setup)
- **Dataset:** Curated conversational dataset optimized for mobile edge constraints
- **Hardware:** Single A100 via RunPod
- **Framework:** Unsloth (2x faster fine-tuning, same as AxioMapper)
### Fine-tuned models (coming soon)
- `pocketllm-llama-1b-v1.gguf` — Llama 3.2 1B fine-tuned for mobile chat
- `pocketllm-gemma-1b-v1.gguf` — Gemma 3 1B fine-tuned for persona consistency
---
## Model Selection Guide
```
Your phone has... Best model to start with
─────────────────────────────────────────────────────
2 GB RAM (budget) → Qwen 2.5 0.5B (fastest)
3 GB RAM → Llama 3.2 1B (balanced speed)
4 GB RAM (mid-range) → Gemma 1.1 2B (best all-rounder)
6 GB RAM → Llama 3.2 3B (better quality)
8 GB RAM (flagship) → Phi-3.5 Mini (best for coding)
```
---
## Technical Details
### MediaPipe format (`.bin`, `.task`)
- Used for Gemma family
- Runs via Google's MediaPipe LLM Inference SDK
- Supports CPU and GPU acceleration (Vulkan/OpenCL)
- Integrated via `react-native-llm-mediapipe`
### GGUF format (`.gguf`)
- Used for Llama, Qwen, Phi, SmolLM, Gemma 3
- Runs via llama.cpp (the gold standard for mobile inference)
- Q4_K_M quantization — best balance of size and quality
- Integrated via `llama.rn`
---
## Repository Structure
```
pocketllm-models/
├── gemma-1.1-2b-it-cpu-int4.bin ← Gemma 2B CPU (MediaPipe)
├── gemma-1.1-2b-it-gpu-int4.bin ← Gemma 2B GPU (MediaPipe)
├── gemma-3-1b-it-q4_k_m.gguf ← Gemma 3 1B (GGUF)
├── gemma-3-4b-it-q4_k_m.gguf ← Gemma 3 4B (GGUF)
├── llama-3.2-1b-instruct-q4_k_m.gguf ← Llama 3.2 1B (GGUF)
├── llama-3.2-3b-instruct-q4_k_m.gguf ← Llama 3.2 3B (GGUF)
├── phi-3.5-mini-instruct-q4_k_m.gguf ← Phi-3.5 Mini (GGUF)
├── qwen2.5-0.5b-instruct-q4_k_m.gguf ← Qwen 2.5 0.5B (GGUF)
└── smollm2-1.7b-instruct-q4_k_m.gguf ← SmolLM2 1.7B (GGUF)
```
---
## License
- **Gemma models:** [Gemma Terms of Use](https://ai.google.dev/gemma/terms)
- **Llama models:** [Meta Llama 3.2 Community License](https://llama.meta.com/llama3_2/license/)
- **Phi models:** [MIT License](https://huggingface.co/microsoft/Phi-3.5-mini-instruct/blob/main/LICENSE)
- **Qwen models:** [Apache 2.0](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct/blob/main/LICENSE)
- **SmolLM2:** [Apache 2.0](https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B-Instruct/blob/main/LICENSE)
- **Fine-tuned models by PocketLLM:** Apache 2.0
---
## Built By
**Amaresh Hebbar**
Building PocketLLM: the only mobile app that runs a full AI agent stack — smart routing, persona memory, MCP tools — completely offline on Android.
*"Your AI. Your phone. Nobody else's business."*
[](https://github.com/amareshhebbar)