README.md

---
license: apache-2.0
base_model: Qwen/Qwen3-8B
tags:
  - rkllm
  - rk3576
  - npu
  - qwen3
---

# Qwen3-8B-Instruct — RK3576 (RKLLM)

RKLLM conversion of Qwen/Qwen3-8B for Rockchip RK3576 NPU inference (e.g. NanoPi R76S, Radxa Rock 4D, Orange Pi with RK3576).

## Conversion details

- Toolkit: rkllm-toolkit v1.2.3
- Target platform: RK3576 (dual-core NPU)
- Quantization: w4a16 (weights 4-bit, activations 16-bit)
- Quantization algorithm: normal
- NPU cores: 2
- Max context: 4096 tokens
- Optimization level: 1

## Files

- Qwen3-8B-rk3576.rkllm — converted model
- tokenizer.json, tokenizer_config.json, vocab.json, merges.txt — original tokenizer files from Qwen/Qwen3-8B

## Usage with RKLLama

Place the .rkllm file and tokenizer files in a model folder with this structure:

Qwen3-8B-Instruct/
├── Modelfile
├── Qwen3-8B-rk3576.rkllm
└── tokenizer/
    ├── tokenizer.json
    ├── tokenizer_config.json
    ├── vocab.json
    └── merges.txt

Modelfile content:

FROM="Qwen3-8B-rk3576.rkllm"
HUGGINGFACE_PATH="Qwen/Qwen3-8B"
TOKENIZER="/models/Qwen3-8B-Instruct/tokenizer"

Restart RKLLama to detect the new model:

docker restart rkllama

## Notes

- Qwen3 is a hybrid thinking model. Depending on your runtime/UI, it may produce think tags with reasoning content.
- Tested on NanoPi R76S (RK3576, 16GB RAM) via RKLLama + Open WebUI.
- License follows the original Qwen3 license (Apache 2.0).