File size: 2,719 Bytes
c12694c | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 | ---
license: apache-2.0
base_model: Qwen/Qwen2.5-1.5B-Instruct
tags:
- rkllm
- rk3588
- npu
- quantized
- w8a8
pipeline_tag: text-generation
---
# Qwen2.5-1.5B-Instruct for RKLLM v1.2.3 (RK3588 NPU)
Pre-converted Qwen2.5-1.5B-Instruct for the Rockchip RK3588 NPU using rknn-llm runtime v1.2.3.
Runs on Orange Pi 5 Plus, Rock 5B, Radxa NX5, and other RK3588-based SBCs with 8GB+ RAM.
## Files
| File | Size | Description |
|------|------|-------------|
| `Qwen2.5-1.5B-Instruct-rk3588-v123-w8a8.rkllm` | 2.0 GB | LLM (W8A8 quantized, 8192 token context) |
## Quick Start
### Directory Structure
```
~/models/Qwen2.5-1.5B-Instruct/
Qwen2.5-1.5B-Instruct-rk3588-v123-w8a8.rkllm
```
### Using with RKLLM API Server
This model is designed for use with the [RKLLM API Server](https://github.com/airockchip/rknn-llm), which provides an OpenAI-compatible API for RK3588 NPU inference. The server auto-discovers `.rkllm` files by scanning subdirectories of your models folder.
```bash
# Place the model in your models directory
mkdir -p ~/models/Qwen2.5-1.5B-Instruct
# Copy .rkllm file here — the API server will find it automatically
sudo systemctl restart rkllm-api
```
The model will appear as `qwen2.5-1.5b-instruct` in the OpenAI-compatible model list.
## Export Details
| Parameter | Value |
|-----------|-------|
| Source | [Qwen/Qwen2.5-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct) |
| Tool | rkllm-toolkit v1.2.3 |
| Quantization | W8A8 (8-bit weights, 8-bit activations) |
| Optimization level | 1 |
| Target platform | rk3588 |
| NPU cores | 3 |
| Max context | 8192 tokens |
## Performance Benchmarks
Tested on Orange Pi 5 Plus (16GB RAM), RK3588 SoC, RKNPU driver 0.9.8:
| Metric | Value |
|--------|-------|
| Decode speed | ~19 tok/s |
| Model load time | ~3 s |
| Peak RAM | ~2.2 GB |
## Known Limitations
- **Tool calling**: Qwen2.5-1.5B is a general-purpose instruct model, not fine-tuned for structured function calling. For agent/tool-calling workloads, see [GatekeeperZA/xLAM-1b-fc-r-RKLLM-v1.2.3](https://huggingface.co/GatekeeperZA/xLAM-1b-fc-r-RKLLM-v1.2.3).
- **Context**: Compiled at 8192 tokens. The runtime will truncate inputs that exceed this limit.
- **Version lock**: This file requires RKLLM runtime v1.2.3 and NPU driver v0.9.8. It is **not compatible** with v1.3.0 or later runtimes.
## License
Apache 2.0, inherited from Qwen2.5-1.5B-Instruct.
## Credits
- Model: [Qwen Team](https://huggingface.co/Qwen) for Qwen2.5-1.5B-Instruct
- Runtime: [Rockchip / airockchip](https://github.com/airockchip/rknn-llm) for rkllm-toolkit
- API Server: [RKLLM API Server](https://github.com/airockchip/rknn-llm) — OpenAI-compatible server for RK3588 NPU
|