| --- |
| license: apache-2.0 |
| base_model: Qwen/Qwen3.5-0.8B |
| tags: |
| - rkllm |
| - rk3588 |
| - rockchip |
| - npu |
| - quantized |
| - qwen3 |
| language: |
| - en |
| - zh |
| --- |
| |
| # Qwen3.5-0.8B — RKLLM v1.2.3 (w8a8, RK3588) |
|
|
| RKLLM conversion of [Qwen/Qwen3.5-0.8B](https://huggingface.co/Qwen/Qwen3.5-0.8B) for Rockchip RK3588 NPU inference. |
|
|
| Converted with RKLLM Toolkit v1.2.3. This is the base model variant — suitable for custom fine-tunes or applications that supply their own system prompting. |
|
|
| ## Key Details |
|
|
| | Property | Value | |
| |----------|-------| |
| | Base Model | Qwen/Qwen3.5-0.8B | |
| | Toolkit Version | RKLLM Toolkit v1.2.3 | |
| | Runtime Version | RKLLM Runtime ≥ v1.2.1 (v1.2.3 recommended) | |
| | Quantization | w8a8 (8-bit weights, 8-bit activations) | |
| | Target Platform | RK3588 | |
| | NPU Cores | 3 | |
| | Thinking Mode | ❌ Not applicable (base model) | |
| | Languages | English, Chinese (multilingual) | |
|
|
| ## Why This Model? |
|
|
| Qwen3.5-0.8B is the successor to Qwen3-0.6B, offering improved architecture and training. At under 1GB after quantisation it is the smallest model in the RK3588 lineup — ideal for low-latency responses, always-on assistants, or edge deployments where memory is tight. |
|
|
| ## Hardware Tested |
|
|
| - **Orange Pi 5 Plus** — RK3588, 16GB RAM, Armbian Linux |
| - RKNPU driver 0.9.8 |
| - RKLLM Runtime v1.2.3 |
|
|
| ## Usage |
|
|
| ### With the RKLLM API Server |
|
|
| ```bash |
| mkdir -p ~/models/Qwen3.5-0.8B |
| cd ~/models/Qwen3.5-0.8B |
| git lfs install && git clone https://huggingface.co/GatekeeperZA/Qwen3.5-0.8B-RKLLM-v1.2.3 . |
| ``` |
|
|
| Use with [GatekeeperZA/RKLLM-API-Server](https://github.com/GatekeeperZA/RKLLM-API-Server). |
|
|
| ### With the official RKLLM API demo |
|
|
| ```bash |
| git clone https://github.com/airockchip/rknn-llm.git |
| cd rknn-llm/examples/rkllm_api_demo |
| ./build/rkllm_api_demo /path/to/Qwen3.5-0.8B-rk3588-w8a8.rkllm 4096 8192 |
| ``` |
|
|
| ## File Listing |
|
|
| | File | Description | |
| |------|-------------| |
| | `Qwen3.5-0.8B-rk3588-w8a8.rkllm` | Quantized model for RK3588 NPU | |
|
|
| ## Compatibility Notes |
|
|
| - Minimum runtime: RKLLM Runtime v1.2.1. v1.2.3 recommended. |
| - RKNPU driver: ≥ 0.9.6 |
| - SoCs: RK3588 / RK3588S. Not compatible with RK3576 without reconversion. |
| - RAM: ~1.5GB loaded. Runs on any RK3588 board including 8GB variants. |
|
|
| ## Acknowledgements |
|
|
| - Alibaba Qwen Team for Qwen3.5 |
| - Rockchip / airockchip for the RKLLM toolkit and runtime |
| - Converted by [GatekeeperZA](https://huggingface.co/GatekeeperZA) |
|
|
