metadata
license: apache-2.0
library_name: rkllm
base_model: Qwen/Qwen2.5-3B-Instruct
tags:
- rkllm
- rk3588
- npu
- rockchip
- quantized
- orange-pi
model_name: Qwen2.5-3B-Instruct-RKLLM
pipeline_tag: text-generation
Qwen2.5-3B-Instruct for RK3588
This repository contains the optimized RKLLM (Language) models for running Qwen2.5-3B-Instruct on Rockchip RK3588 NPUs.
📦 Files
| File | Type | Description |
|---|---|---|
qwen2.5-3b-instruct_w8a8_rk3588.rkllm |
RKLLM | Language Decoder (w8a8) |
⚙️ System Requirements
- NPU Driver:
≥ 0.9.8 - RKLLM Runtime:
v1.2.3 - RKNN-Toolkit2:
v2.3.0
💻 Tested Hardware
- Device: Orange Pi 5 Pro
- RAM: 16GB
- OS: Armbian OS
🙏 Acknowledgements
- Qwen Team for the base model architecture.
- Rockchip / airockchip for the RKLLM toolkit and runtime.
- Converted by Akmal Dira.