---
language:
- en
- zh
license: apache-2.0
tags:
- mobile
- edge-ai
- code
- quantized
- gguf
- small-language-model
pipeline_tag: text-generation
---

# Qwen 2.5 0.5B Instruct - Mobile INT4 (GGUF)

**Alibaba's Qwen 2.5 0.5B Instruct**, the smallest capable general-purpose model. Incredibly fast on phones.

| Property | Value |
|----------|-------|
| **Base** | Qwen/Qwen2.5-0.5B-Instruct |
| **Parameters** | 494 million |
| **Quantization** | INT4 GGUF |
| **Size** | ~398 MB |
| **License** | Apache 2.0 |

## Performance

- **~45 tok/s** on Samsung S20 FE CPU (fastest in our collection!)
- ~0.7 GB memory footprint
- Fits on ANY modern smartphone
- ~94% quality retention

## Use Cases

- Code generation on mobile IDEs
- Quick text classification / extraction
- Embedded assistants in apps
- Ultra-low-latency responses (<50ms per token)
- Batch processing at massive scale

## Quick Start

```bash
huggingface-cli download dispatchAI/Qwen2.5-0.5B-Instruct-mobile-int4 --local-dir ./models
./build/bin/main -m ./models/model.gguf -p "Explain quantum computing simply." -n 128 -t 4
```