Qwen3-8B-Instruct — RK3576 (RKLLM)
RKLLM conversion of Qwen/Qwen3-8B for Rockchip RK3576 NPU inference (e.g. NanoPi R76S, Radxa Rock 4D, Orange Pi with RK3576).
Conversion details
- Toolkit: rkllm-toolkit v1.2.3
- Target platform: RK3576 (dual-core NPU)
- Quantization: w4a16 (weights 4-bit, activations 16-bit)
- Quantization algorithm: normal
- NPU cores: 2
- Max context: 4096 tokens
- Optimization level: 1
Files
- Qwen3-8B-rk3576.rkllm — converted model
- tokenizer.json, tokenizer_config.json, vocab.json, merges.txt — original tokenizer files from Qwen/Qwen3-8B
Usage with RKLLama
Place the .rkllm file and tokenizer files in a model folder with this structure:
Qwen3-8B-Instruct/ ├── Modelfile ├── Qwen3-8B-rk3576.rkllm └── tokenizer/ ├── tokenizer.json ├── tokenizer_config.json ├── vocab.json └── merges.txt
Modelfile content:
FROM="Qwen3-8B-rk3576.rkllm" HUGGINGFACE_PATH="Qwen/Qwen3-8B" TOKENIZER="/models/Qwen3-8B-Instruct/tokenizer"
Restart RKLLama to detect the new model:
docker restart rkllama
Notes
- Qwen3 is a hybrid thinking model. Depending on your runtime/UI, it may produce think tags with reasoning content.
- Tested on NanoPi R76S (RK3576, 16GB RAM) via RKLLama + Open WebUI.
- License follows the original Qwen3 license (Apache 2.0).
- Downloads last month
- 22
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js