license: apache-2.0
base_model: Qwen/Qwen3-1.7B
library_name: rkllm
pipeline_tag: text-generation
tags:
- rkllm
- rockchip
- rk3576
- rk3588
- qwen
- qwen3
Qwen3-1.7B-RKLLM
RKLLM-converted versions of Qwen3-1.7B for deployment on supported Rockchip NPUs.
.rkllm files require the Rockchip RKLLM runtime and cannot be loaded directly with Transformers, llama.cpp, or Ollama.
Base Model
- Model: Qwen3-1.7B
- Author: Alibaba Cloud Qwen Team
- Original Model: Qwen/Qwen3-1.7B
- Original License: Apache-2.0
Refer to the upstream model card for the original model’s capabilities and limitations.
Supported Platforms
| SoC | Quantization | Status |
|---|---|---|
| RK3576 | W4A16 | ✅ Tested |
| RK3576 | W8A8 | ✅ Tested |
| RK3588 | W8A8 | ✅ Supported |
RK3576 W4A16 and W8A8 are tested. RK3588 W8A8 is supported as a target platform but is not marked as tested.
Conversion
- RKLLM Toolkit: 1.2.3
- Target Platforms: RK3576, RK3588
Available Variants
| File | Quantization |
|---|---|
| Qwen3-1.7B_w4a16_RK3576.rkllm | W4A16 |
| Qwen3-1.7B_w8a8_RK3576.rkllm | W8A8 |
| Qwen3-1.7B_w8a8_RK3588.rkllm | W8A8 |
Repository Structure
.
├── RK3576
│ ├── Qwen3-1.7B_w4a16_RK3576.rkllm
│ └── Qwen3-1.7B_w8a8_RK3576.rkllm
└── RK3588
└── Qwen3-1.7B_w8a8_RK3588.rkllm
Usage
Download the desired RKLLM model and deploy it with the RKLLM runtime on the matching Rockchip platform.
For faster deployment without setting up the environment manually, see Hanzo-Huang/rkllm-docker.
Docker Deployment
Use the model-specific image for the fastest RK3576 W4A16 deployment. The model is included in the image:
sudo docker run --rm -it \
--privileged \
-p 8001:8001 \
-v /dev:/dev \
ghcr.io/hanzo-huang/rkllm-docker/qwen3-1.7b:w4a16-rk3576
Alternatively, use the reusable environment image and mount a downloaded model:
sudo docker run --rm -it \
--privileged \
-p 8001:8001 \
-v /dev:/dev \
-v ./:/app/models:ro \
-e MODEL_PATH=/app/models/RK3576/Qwen3-1.7B_w4a16_RK3576.rkllm \
-e TARGET_PLATFORM=rk3576 \
ghcr.io/hanzo-huang/rkllm-docker:env-latest
The second command requires downloading the selected .rkllm file first. Use the matching model variant and target platform.
License
The original Qwen3-1.7B model is released under the Apache-2.0 license. Converted artifacts are derived from the original model and remain subject to the applicable upstream license terms.
Acknowledgements
Thanks to the Qwen Team for releasing the original model and to Rockchip and RKLLM contributors for the deployment toolchain.