README.md

---
license: apache-2.0
tags:
- pocketllm
- on-device
- edge-ai
- mobile
- android
- offline
- mediapipe
- gguf
- llama
- gemma
- qwen
- phi
language:
- en
- hi
- multilingual
pipeline_tag: text-generation
---

# PocketLLM Model Collection

<div align="center">

**The official model collection for [PocketLLM](https://github.com/amareshhebbar/PocketLLM)**

*Private • Offline • On-Device AI for Android*

[![GitHub](https://img.shields.io/badge/GitHub-PocketLLM-black?logo=github)](https://github.com/amareshhebbar/PocketLLM)
[![Platform](https://img.shields.io/badge/Platform-Android-green?logo=android)](https://github.com/amareshhebbar/PocketLLM)
[![License](https://img.shields.io/badge/License-Apache%202.0-blue)](LICENSE)

</div>

---

## What is PocketLLM?

PocketLLM is an Android app that runs large language models **completely on your device** — no internet required during inference, no data sent to the cloud, no subscriptions.

This repository hosts the curated model collection optimized for mobile edge inference using:
- **MediaPipe LLM Inference** (for `.bin` and `.task` files — Gemma family)
- **llama.cpp via llama.rn** (for `.gguf` files — Llama, Phi, Qwen, SmolLM, Gemma 3)

---

## Model Catalog

### ⚡ Ultra Fast — Under 1 GB, fits any Android phone

| Model | File | Size | Format | RAM |
|-------|------|------|--------|-----|
| Qwen 2.5 0.5B | `qwen2.5-0.5b-instruct-q4_k_m.gguf` | 0.4 GB | GGUF | 2 GB |
| Gemma 3 1B | `gemma-3-1b-it-q4_k_m.gguf` | 0.7 GB | GGUF | 2 GB |
| Llama 3.2 1B | `llama-3.2-1b-instruct-q4_k_m.gguf` | 0.8 GB | GGUF | 3 GB |
| SmolLM2 1.7B | `smollm2-1.7b-instruct-q4_k_m.gguf` | 1.0 GB | GGUF | 3 GB |

### ⚖️ Balanced — 1–2 GB, great quality on mid-range phones

| Model | File | Size | Format | RAM |
|-------|------|------|--------|-----|
| Gemma 1.1 2B (CPU) | `gemma-1.1-2b-it-cpu-int4.bin` | 1.35 GB | MediaPipe | 4 GB |
| Gemma 1.1 2B (GPU) | `gemma-1.1-2b-it-gpu-int4.bin` | 1.35 GB | MediaPipe | 4 GB |
| Llama 3.2 3B | `llama-3.2-3b-instruct-q4_k_m.gguf` | 2.0 GB | GGUF | 4 GB |

### 🚀 Powerful — Best quality, needs 5GB+ RAM

| Model | File | Size | Format | RAM |
|-------|------|------|--------|-----|
| Phi-3.5 Mini | `phi-3.5-mini-instruct-q4_k_m.gguf` | 2.4 GB | GGUF | 5 GB |
| Gemma 3 4B | `gemma-3-4b-it-q4_k_m.gguf` | 2.8 GB | GGUF | 6 GB |

---

## How to Use in PocketLLM

Models are downloaded directly inside the PocketLLM app. Open the **Model Store** tab, select a model, and tap Download. The app handles everything automatically.

**Direct download URLs:**
```
https://huggingface.co/AmareshHebbar/pocketllm-models/resolve/main/<filename>
```

---

## Fine-Tuning Roadmap

> *Coming soon — within the next 2 weeks*

We are fine-tuning these base models specifically for **on-device conversational AI** on mobile:

### Goals
- Better persona adherence (the model stays in character consistently)
- Shorter, more natural responses (base models tend to be verbose on mobile)
- Improved memory utilization (uses injected memories naturally)
- Better instruction following for small context windows (2048 tokens)
- Hindi + English code-switching support (for India market)

### Training approach
- **Base:** Llama 3.2 1B, Gemma 3 1B (smallest models first — fastest iteration)
- **Method:** QLoRA fine-tuning (4-bit, similar to the AxioMapper training setup)
- **Dataset:** Curated conversational dataset optimized for mobile edge constraints
- **Hardware:** Single A100 via RunPod
- **Framework:** Unsloth (2x faster fine-tuning, same as AxioMapper)

### Fine-tuned models (coming soon)
- `pocketllm-llama-1b-v1.gguf` — Llama 3.2 1B fine-tuned for mobile chat
- `pocketllm-gemma-1b-v1.gguf` — Gemma 3 1B fine-tuned for persona consistency

---

## Model Selection Guide

```
Your phone has...          Best model to start with
─────────────────────────────────────────────────────
2 GB RAM (budget)    →    Qwen 2.5 0.5B  (fastest)
3 GB RAM             →    Llama 3.2 1B   (balanced speed)
4 GB RAM (mid-range) →    Gemma 1.1 2B   (best all-rounder)
6 GB RAM             →    Llama 3.2 3B   (better quality)
8 GB RAM (flagship)  →    Phi-3.5 Mini   (best for coding)
```

---

## Technical Details

### MediaPipe format (`.bin`, `.task`)
- Used for Gemma family
- Runs via Google's MediaPipe LLM Inference SDK
- Supports CPU and GPU acceleration (Vulkan/OpenCL)
- Integrated via `react-native-llm-mediapipe`

### GGUF format (`.gguf`)
- Used for Llama, Qwen, Phi, SmolLM, Gemma 3
- Runs via llama.cpp (the gold standard for mobile inference)
- Q4_K_M quantization — best balance of size and quality
- Integrated via `llama.rn`

---

## Repository Structure

```
pocketllm-models/
├── gemma-1.1-2b-it-cpu-int4.bin       ← Gemma 2B CPU (MediaPipe)
├── gemma-1.1-2b-it-gpu-int4.bin       ← Gemma 2B GPU (MediaPipe)
├── gemma-3-1b-it-q4_k_m.gguf         ← Gemma 3 1B (GGUF)
├── gemma-3-4b-it-q4_k_m.gguf         ← Gemma 3 4B (GGUF)
├── llama-3.2-1b-instruct-q4_k_m.gguf ← Llama 3.2 1B (GGUF)
├── llama-3.2-3b-instruct-q4_k_m.gguf ← Llama 3.2 3B (GGUF)
├── phi-3.5-mini-instruct-q4_k_m.gguf ← Phi-3.5 Mini (GGUF)
├── qwen2.5-0.5b-instruct-q4_k_m.gguf ← Qwen 2.5 0.5B (GGUF)
└── smollm2-1.7b-instruct-q4_k_m.gguf ← SmolLM2 1.7B (GGUF)
```

---

## License

- **Gemma models:** [Gemma Terms of Use](https://ai.google.dev/gemma/terms)
- **Llama models:** [Meta Llama 3.2 Community License](https://llama.meta.com/llama3_2/license/)
- **Phi models:** [MIT License](https://huggingface.co/microsoft/Phi-3.5-mini-instruct/blob/main/LICENSE)
- **Qwen models:** [Apache 2.0](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct/blob/main/LICENSE)
- **SmolLM2:** [Apache 2.0](https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B-Instruct/blob/main/LICENSE)
- **Fine-tuned models by PocketLLM:** Apache 2.0

---

## Built By

**Amaresh Hebbar**

Building PocketLLM: the only mobile app that runs a full AI agent stack — smart routing, persona memory, MCP tools — completely offline on Android.

*"Your AI. Your phone. Nobody else's business."*

[![GitHub](https://img.shields.io/badge/GitHub-amareshhebbar-black?logo=github)](https://github.com/amareshhebbar)