| license: apache-2.0 | |
| library_name: rkllm | |
| base_model: Qwen/Qwen2.5-3B-Instruct | |
| tags: | |
| - rkllm | |
| - rk3588 | |
| - npu | |
| - rockchip | |
| - quantized | |
| - orange-pi | |
| model_name: Qwen2.5-3B-Instruct-RKLLM | |
| pipeline_tag: text-generation | |
| # Qwen2.5-3B-Instruct for RK3588 | |
| This repository contains the optimized **RKLLM** (Language) models for running **Qwen2.5-3B-Instruct** on Rockchip RK3588 NPUs. | |
| ### 📦 Files | |
| | File | Type | Description | | |
| | --- | --- | --- | | |
| | `qwen2.5-3b-instruct_w8a8_rk3588.rkllm` | **RKLLM** | Language Decoder (w8a8) | | |
| --- | |
| ### ⚙️ System Requirements | |
| * **NPU Driver:** `≥ 0.9.8` | |
| * **RKLLM Runtime:** `v1.2.3` | |
| * **RKNN-Toolkit2:** `v2.3.0` | |
| ### 💻 Tested Hardware | |
| * **Device:** Orange Pi 5 Pro | |
| * **RAM:** 16GB | |
| * **OS:** Armbian OS | |
| --- | |
| ### 🙏 Acknowledgements | |
| * [Qwen Team](https://huggingface.co/qwen) for the base model architecture. | |
| * [Rockchip / airockchip](https://github.com/airockchip/rknn-llm) for the RKLLM toolkit and runtime. | |
| * Converted by [Akmal Dira](https://huggingface.co/akmaldira). | |
| --- | |