File size: 2,395 Bytes
59a42e2 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 | ---
license: apache-2.0
base_model: Qwen/Qwen3-Reranker-0.6B
tags:
- rkllm
- rk3588
- rockchip
- npu
- quantized
- qwen3
- reranker
- text-ranking
language:
- en
- zh
pipeline_tag: text-ranking
---
# Qwen3-Reranker-0.6B — RKLLM v1.2.3 (w8a8, RK3588)
RKLLM conversion of [Qwen/Qwen3-Reranker-0.6B](https://huggingface.co/Qwen/Qwen3-Reranker-0.6B) for Rockchip RK3588 NPU inference.
Converted with RKLLM Toolkit v1.2.3. This model re-ranks retrieved documents for RAG pipelines, improving result quality over pure vector-similarity search.
## Key Details
| Property | Value |
|----------|-------|
| Base Model | Qwen/Qwen3-Reranker-0.6B |
| Toolkit Version | RKLLM Toolkit v1.2.3 |
| Runtime Version | RKLLM Runtime ≥ v1.2.1 (v1.2.3 recommended) |
| Quantization | w8a8 (8-bit weights, 8-bit activations) |
| Target Platform | RK3588 |
| NPU Cores | 3 |
| Optimization Level | 1 |
| Hybrid Ratio | 0.5 |
| Model Type | Reranker (cross-encoder) |
| Languages | English, Chinese (multilingual) |
## Why This Model?
A reranker is the second stage of a RAG retrieval pipeline — after the embedding model retrieves candidates, the reranker scores each (query, document) pair more precisely. Running it on the RK3588 NPU keeps the full pipeline local and GPU-free.
Pair with the Qwen3-Embedding-0.6B for a complete local retrieval stack.
## Hardware Tested
- **Orange Pi 5 Plus** — RK3588, 16GB RAM, Armbian Linux
- RKNPU driver 0.9.8
- RKLLM Runtime v1.2.3
## Usage
### With the RKLLM Reranker Service
```bash
mkdir -p ~/models/Qwen3-Reranker-0.6B
cd ~/models/Qwen3-Reranker-0.6B
git lfs install && git clone https://huggingface.co/GatekeeperZA/Qwen3-Reranker-0.6B-RKLLM-v1.2.3 .
```
Use with [GatekeeperZA/RKLLM-API-Server](https://github.com/GatekeeperZA/RKLLM-API-Server) reranker endpoint.
## File Listing
| File | Description |
|------|-------------|
| `Qwen3-Reranker-0.6B-rk3588-w8a8-opt-1-hybrid-ratio-0.5.rkllm` | Quantized reranker model for RK3588 NPU |
## Compatibility Notes
- Minimum runtime: RKLLM Runtime v1.2.1. v1.2.3 recommended.
- RKNPU driver: ≥ 0.9.6
- SoCs: RK3588 / RK3588S. Not compatible with RK3576 without reconversion.
- RAM: ~1GB loaded.
## Acknowledgements
- Alibaba Qwen Team for Qwen3-Reranker
- Rockchip / airockchip for the RKLLM toolkit and runtime
- Converted by [GatekeeperZA](https://huggingface.co/GatekeeperZA)
|