README.md

metadata

license: apache-2.0
base_model: Qwen/Qwen3-8B
tags:
  - rkllm
  - rk3576
  - npu
  - qwen3

Qwen3-8B-Instruct — RK3576 (RKLLM)

RKLLM conversion of Qwen/Qwen3-8B for Rockchip RK3576 NPU inference (e.g. NanoPi R76S, Radxa Rock 4D, Orange Pi with RK3576).

Conversion details

• Toolkit: rkllm-toolkit v1.2.3
• Target platform: RK3576 (dual-core NPU)
• Quantization: w4a16 (weights 4-bit, activations 16-bit)
• Quantization algorithm: normal
• NPU cores: 2
• Max context: 4096 tokens
• Optimization level: 1

Files

• Qwen3-8B-rk3576.rkllm — converted model
• tokenizer.json, tokenizer_config.json, vocab.json, merges.txt — original tokenizer files from Qwen/Qwen3-8B

Usage with RKLLama

Place the .rkllm file and tokenizer files in a model folder with this structure:

Qwen3-8B-Instruct/ ├── Modelfile ├── Qwen3-8B-rk3576.rkllm └── tokenizer/ ├── tokenizer.json ├── tokenizer_config.json ├── vocab.json └── merges.txt

Modelfile content:

FROM="Qwen3-8B-rk3576.rkllm" HUGGINGFACE_PATH="Qwen/Qwen3-8B" TOKENIZER="/models/Qwen3-8B-Instruct/tokenizer"

Restart RKLLama to detect the new model:

docker restart rkllama

Notes

• Qwen3 is a hybrid thinking model. Depending on your runtime/UI, it may produce think tags with reasoning content.
• Tested on NanoPi R76S (RK3576, 16GB RAM) via RKLLama + Open WebUI.
• License follows the original Qwen3 license (Apache 2.0).