Instructions to use HanzoHuang/Qwen3.5-4B-RKLLM with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- RKLLM
How to use HanzoHuang/Qwen3.5-4B-RKLLM with RKLLM:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Document RKLLM artifacts and checksums
Browse files
README.md
CHANGED
|
@@ -15,32 +15,31 @@ tags:
|
|
| 15 |
|
| 16 |
# Qwen3.5-4B-RKLLM
|
| 17 |
|
| 18 |
-
RKLLM-converted Qwen3.5-4B artifacts for
|
| 19 |
|
| 20 |
-
`.rkllm` and `.rknn`
|
| 21 |
|
| 22 |
-
## Base
|
| 23 |
|
| 24 |
-
-
|
| 25 |
-
-
|
| 26 |
-
-
|
| 27 |
-
- Original license: Apache-2.0
|
| 28 |
|
| 29 |
-
|
| 30 |
|
| 31 |
-
|
| 32 |
|
| 33 |
-
| Target |
|
| 34 |
-
| --- | --- | --- | --- |
|
| 35 |
-
| RK3576 | [
|
| 36 |
-
| RK3576 | [
|
| 37 |
-
| RK3588 | [
|
| 38 |
|
| 39 |
-
The
|
| 40 |
|
| 41 |
## Usage
|
| 42 |
|
| 43 |
-
Download
|
| 44 |
|
| 45 |
```bash
|
| 46 |
hf download HanzoHuang/Qwen3.5-4B-RKLLM \
|
|
@@ -49,14 +48,12 @@ hf download HanzoHuang/Qwen3.5-4B-RKLLM \
|
|
| 49 |
--local-dir Qwen3.5-4B-RKLLM
|
| 50 |
```
|
| 51 |
|
| 52 |
-
|
| 53 |
|
| 54 |
## Limitations
|
| 55 |
|
| 56 |
-
|
| 57 |
-
- Runtime, driver, and toolkit compatibility depends on the Rockchip software stack installed on the device.
|
| 58 |
-
- Conversion may change output quality relative to the upstream floating-point model; validate on your own workload.
|
| 59 |
|
| 60 |
## Acknowledgements
|
| 61 |
|
| 62 |
-
Thanks to the Qwen Team
|
|
|
|
| 15 |
|
| 16 |
# Qwen3.5-4B-RKLLM
|
| 17 |
|
| 18 |
+
RKLLM/RKNN-converted Qwen3.5-4B multimodal artifacts for Rockchip RK3576 and RK3588 NPUs.
|
| 19 |
|
| 20 |
+
This is a VLM conversion: each supported platform requires both the `.rkllm` language model and the matching `.rknn` vision encoder. The pair must come from the same platform directory. These are hardware-specific artifacts, not Transformers checkpoints.
|
| 21 |
|
| 22 |
+
## Base model
|
| 23 |
|
| 24 |
+
- Upstream model: [Qwen/Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B)
|
| 25 |
+
- License: Apache-2.0
|
| 26 |
+
- Model type: VLM (vision-language model)
|
|
|
|
| 27 |
|
| 28 |
+
## Conversion and variants
|
| 29 |
|
| 30 |
+
Converted with RKLLM Toolkit 1.3.0 and an RKNN vision conversion. Use a matching pair for the exact target SoC.
|
| 31 |
|
| 32 |
+
| Target | Quantization | RKLLM language model | RKLLM SHA256 | RKNN vision encoder | RKNN SHA256 |
|
| 33 |
+
| --- | --- | --- | --- | --- | --- |
|
| 34 |
+
| RK3576 | W4A16 (g128) | [Qwen3.5-4B_RK3576_w4a16_g128.rkllm](RK3576/Qwen3.5-4B_RK3576_w4a16_g128.rkllm) | `aa4d34b42752a0e491ed891ce4b7f32a745631601a65b175f743f971bdb33482` | [Qwen3.5-4B_vision_RK3576.rknn](RK3576/Qwen3.5-4B_vision_RK3576.rknn) | `6692f52adaa7ee0ba892c7cbba01979750cb82cff8061a571f56e9d84c6d98a2` |
|
| 35 |
+
| RK3576 | W8A8 | [Qwen3.5-4B_RK3576_w8a8.rkllm](RK3576/Qwen3.5-4B_RK3576_w8a8.rkllm) | `736f4b1065b481bae9cbd86e6c9ed30222abba9c5a3cb2d8d68dd92037d82dfe` | [Qwen3.5-4B_vision_RK3576.rknn](RK3576/Qwen3.5-4B_vision_RK3576.rknn) | `6692f52adaa7ee0ba892c7cbba01979750cb82cff8061a571f56e9d84c6d98a2` |
|
| 36 |
+
| RK3588 | W8A8 | [Qwen3.5-4B_RK3588_w8a8.rkllm](RK3588/Qwen3.5-4B_RK3588_w8a8.rkllm) | `715566bbee72b25d8c4912f6cf6256a8ac264a754511573a99a541487bbc06b4` | [Qwen3.5-4B_vision_RK3588.rknn](RK3588/Qwen3.5-4B_vision_RK3588.rknn) | `c286ef69266c11a2a2cedd881ad6a40415c17c8924962f07454359bb6d60e2ba` |
|
| 37 |
|
| 38 |
+
The root `Qwen3.5-4B_vision.onnx` is the vision conversion input; use the platform-specific `.rknn` encoder for deployment.
|
| 39 |
|
| 40 |
## Usage
|
| 41 |
|
| 42 |
+
Download both files for the target platform:
|
| 43 |
|
| 44 |
```bash
|
| 45 |
hf download HanzoHuang/Qwen3.5-4B-RKLLM \
|
|
|
|
| 48 |
--local-dir Qwen3.5-4B-RKLLM
|
| 49 |
```
|
| 50 |
|
| 51 |
+
Use them with the RKLLM VLM runtime. For a Docker API, see [Hanzo-Huang/rkllm-docker](https://github.com/Hanzo-Huang/rkllm-docker) and set `MODEL_KIND=vlm` with both model files.
|
| 52 |
|
| 53 |
## Limitations
|
| 54 |
|
| 55 |
+
The vision encoder and language model are SoC-specific and must be kept as a matching pair. Validate image preprocessing, memory use, and runtime compatibility on your device.
|
|
|
|
|
|
|
| 56 |
|
| 57 |
## Acknowledgements
|
| 58 |
|
| 59 |
+
Thanks to the Qwen Team, Rockchip, and the RKLLM/RKNN community.
|