HanzoHuang commited on
Commit
9e3fdc3
·
verified ·
1 Parent(s): 90dedc0

Document RKLLM artifacts and checksums

Browse files
Files changed (1) hide show
  1. README.md +18 -21
README.md CHANGED
@@ -15,32 +15,31 @@ tags:
15
 
16
  # Qwen3.5-4B-RKLLM
17
 
18
- RKLLM-converted Qwen3.5-4B artifacts for deployment on Rockchip RK3576 and RK3588 NPUs. This repository includes language-model binaries and the matching vision encoder artifacts for multimodal inference.
19
 
20
- `.rkllm` and `.rknn` files require the Rockchip RKLLM/RKNN runtime. They cannot be loaded directly with Transformers, llama.cpp, or Ollama.
21
 
22
- ## Base Model
23
 
24
- - Model: Qwen3.5-4B
25
- - Author: Qwen Team
26
- - Original model: [Qwen/Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B)
27
- - Original license: Apache-2.0
28
 
29
- Refer to the upstream model card for the original model's capabilities, limitations, and acceptable-use guidance.
30
 
31
- ## Available Artifacts
32
 
33
- | Target | Language model | Quantization | Vision model |
34
- | --- | --- | --- | --- |
35
- | RK3576 | [`Qwen3.5-4B_RK3576_w4a16_g128.rkllm`](RK3576/Qwen3.5-4B_RK3576_w4a16_g128.rkllm) | W4A16 (g128) | [`Qwen3.5-4B_vision_RK3576.rknn`](RK3576/Qwen3.5-4B_vision_RK3576.rknn) |
36
- | RK3576 | [`Qwen3.5-4B_RK3576_w8a8.rkllm`](RK3576/Qwen3.5-4B_RK3576_w8a8.rkllm) | W8A8 | [`Qwen3.5-4B_vision_RK3576.rknn`](RK3576/Qwen3.5-4B_vision_RK3576.rknn) |
37
- | RK3588 | [`Qwen3.5-4B_RK3588_w8a8.rkllm`](RK3588/Qwen3.5-4B_RK3588_w8a8.rkllm) | W8A8 | [`Qwen3.5-4B_vision_RK3588.rknn`](RK3588/Qwen3.5-4B_vision_RK3588.rknn) |
38
 
39
- The repository also contains `Qwen3.5-4B_vision.onnx`, the vision encoder in ONNX format, and `Qwen3.5-4B_data_quant.json`, the calibration data used during conversion.
40
 
41
  ## Usage
42
 
43
- Download one language-model artifact and the matching vision artifact for your SoC. Do not mix RK3576 and RK3588 files.
44
 
45
  ```bash
46
  hf download HanzoHuang/Qwen3.5-4B-RKLLM \
@@ -49,14 +48,12 @@ hf download HanzoHuang/Qwen3.5-4B-RKLLM \
49
  --local-dir Qwen3.5-4B-RKLLM
50
  ```
51
 
52
- Load the `.rkllm` model with a compatible RKLLM runtime and the `.rknn` vision encoder with the matching RKNN runtime. Application code must implement Qwen3.5's multimodal preprocessing and prompt format.
53
 
54
  ## Limitations
55
 
56
- - These are hardware-specific converted artifacts, not Transformers checkpoints.
57
- - Runtime, driver, and toolkit compatibility depends on the Rockchip software stack installed on the device.
58
- - Conversion may change output quality relative to the upstream floating-point model; validate on your own workload.
59
 
60
  ## Acknowledgements
61
 
62
- Thanks to the Qwen Team for releasing Qwen3.5-4B and to Rockchip and RKLLM contributors for the deployment toolchain.
 
15
 
16
  # Qwen3.5-4B-RKLLM
17
 
18
+ RKLLM/RKNN-converted Qwen3.5-4B multimodal artifacts for Rockchip RK3576 and RK3588 NPUs.
19
 
20
+ This is a VLM conversion: each supported platform requires both the `.rkllm` language model and the matching `.rknn` vision encoder. The pair must come from the same platform directory. These are hardware-specific artifacts, not Transformers checkpoints.
21
 
22
+ ## Base model
23
 
24
+ - Upstream model: [Qwen/Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B)
25
+ - License: Apache-2.0
26
+ - Model type: VLM (vision-language model)
 
27
 
28
+ ## Conversion and variants
29
 
30
+ Converted with RKLLM Toolkit 1.3.0 and an RKNN vision conversion. Use a matching pair for the exact target SoC.
31
 
32
+ | Target | Quantization | RKLLM language model | RKLLM SHA256 | RKNN vision encoder | RKNN SHA256 |
33
+ | --- | --- | --- | --- | --- | --- |
34
+ | RK3576 | W4A16 (g128) | [Qwen3.5-4B_RK3576_w4a16_g128.rkllm](RK3576/Qwen3.5-4B_RK3576_w4a16_g128.rkllm) | `aa4d34b42752a0e491ed891ce4b7f32a745631601a65b175f743f971bdb33482` | [Qwen3.5-4B_vision_RK3576.rknn](RK3576/Qwen3.5-4B_vision_RK3576.rknn) | `6692f52adaa7ee0ba892c7cbba01979750cb82cff8061a571f56e9d84c6d98a2` |
35
+ | RK3576 | W8A8 | [Qwen3.5-4B_RK3576_w8a8.rkllm](RK3576/Qwen3.5-4B_RK3576_w8a8.rkllm) | `736f4b1065b481bae9cbd86e6c9ed30222abba9c5a3cb2d8d68dd92037d82dfe` | [Qwen3.5-4B_vision_RK3576.rknn](RK3576/Qwen3.5-4B_vision_RK3576.rknn) | `6692f52adaa7ee0ba892c7cbba01979750cb82cff8061a571f56e9d84c6d98a2` |
36
+ | RK3588 | W8A8 | [Qwen3.5-4B_RK3588_w8a8.rkllm](RK3588/Qwen3.5-4B_RK3588_w8a8.rkllm) | `715566bbee72b25d8c4912f6cf6256a8ac264a754511573a99a541487bbc06b4` | [Qwen3.5-4B_vision_RK3588.rknn](RK3588/Qwen3.5-4B_vision_RK3588.rknn) | `c286ef69266c11a2a2cedd881ad6a40415c17c8924962f07454359bb6d60e2ba` |
37
 
38
+ The root `Qwen3.5-4B_vision.onnx` is the vision conversion input; use the platform-specific `.rknn` encoder for deployment.
39
 
40
  ## Usage
41
 
42
+ Download both files for the target platform:
43
 
44
  ```bash
45
  hf download HanzoHuang/Qwen3.5-4B-RKLLM \
 
48
  --local-dir Qwen3.5-4B-RKLLM
49
  ```
50
 
51
+ Use them with the RKLLM VLM runtime. For a Docker API, see [Hanzo-Huang/rkllm-docker](https://github.com/Hanzo-Huang/rkllm-docker) and set `MODEL_KIND=vlm` with both model files.
52
 
53
  ## Limitations
54
 
55
+ The vision encoder and language model are SoC-specific and must be kept as a matching pair. Validate image preprocessing, memory use, and runtime compatibility on your device.
 
 
56
 
57
  ## Acknowledgements
58
 
59
+ Thanks to the Qwen Team, Rockchip, and the RKLLM/RKNN community.