| --- |
| license: apache-2.0 |
| base_model: Qwen/Qwen3-Reranker-0.6B |
| tags: |
| - rkllm |
| - rk3588 |
| - rockchip |
| - npu |
| - quantized |
| - qwen3 |
| - reranker |
| - text-ranking |
| language: |
| - en |
| - zh |
| pipeline_tag: text-ranking |
| --- |
| |
| # Qwen3-Reranker-0.6B — RKLLM v1.2.3 (w8a8, RK3588) |
|
|
| RKLLM conversion of [Qwen/Qwen3-Reranker-0.6B](https://huggingface.co/Qwen/Qwen3-Reranker-0.6B) for Rockchip RK3588 NPU inference. |
|
|
| Converted with RKLLM Toolkit v1.2.3. This model re-ranks retrieved documents for RAG pipelines, improving result quality over pure vector-similarity search. |
|
|
| ## Key Details |
|
|
| | Property | Value | |
| |----------|-------| |
| | Base Model | Qwen/Qwen3-Reranker-0.6B | |
| | Toolkit Version | RKLLM Toolkit v1.2.3 | |
| | Runtime Version | RKLLM Runtime ≥ v1.2.1 (v1.2.3 recommended) | |
| | Quantization | w8a8 (8-bit weights, 8-bit activations) | |
| | Target Platform | RK3588 | |
| | NPU Cores | 3 | |
| | Optimization Level | 1 | |
| | Hybrid Ratio | 0.5 | |
| | Model Type | Reranker (cross-encoder) | |
| | Languages | English, Chinese (multilingual) | |
|
|
| ## Why This Model? |
|
|
| A reranker is the second stage of a RAG retrieval pipeline — after the embedding model retrieves candidates, the reranker scores each (query, document) pair more precisely. Running it on the RK3588 NPU keeps the full pipeline local and GPU-free. |
|
|
| Pair with the Qwen3-Embedding-0.6B for a complete local retrieval stack. |
|
|
| ## Hardware Tested |
|
|
| - **Orange Pi 5 Plus** — RK3588, 16GB RAM, Armbian Linux |
| - RKNPU driver 0.9.8 |
| - RKLLM Runtime v1.2.3 |
|
|
| ## Usage |
|
|
| ### With the RKLLM Reranker Service |
|
|
| ```bash |
| mkdir -p ~/models/Qwen3-Reranker-0.6B |
| cd ~/models/Qwen3-Reranker-0.6B |
| git lfs install && git clone https://huggingface.co/GatekeeperZA/Qwen3-Reranker-0.6B-RKLLM-v1.2.3 . |
| ``` |
|
|
| Use with [GatekeeperZA/RKLLM-API-Server](https://github.com/GatekeeperZA/RKLLM-API-Server) reranker endpoint. |
|
|
| ## File Listing |
|
|
| | File | Description | |
| |------|-------------| |
| | `Qwen3-Reranker-0.6B-rk3588-w8a8-opt-1-hybrid-ratio-0.5.rkllm` | Quantized reranker model for RK3588 NPU | |
|
|
| ## Compatibility Notes |
|
|
| - Minimum runtime: RKLLM Runtime v1.2.1. v1.2.3 recommended. |
| - RKNPU driver: ≥ 0.9.6 |
| - SoCs: RK3588 / RK3588S. Not compatible with RK3576 without reconversion. |
| - RAM: ~1GB loaded. |
|
|
| ## Acknowledgements |
|
|
| - Alibaba Qwen Team for Qwen3-Reranker |
| - Rockchip / airockchip for the RKLLM toolkit and runtime |
| - Converted by [GatekeeperZA](https://huggingface.co/GatekeeperZA) |
|
|
