intent-rk3588-npu

Pre-converted NPU weights for the intent home-robot intent-recognition pipeline on Rockchip RK3588 boards.

File	Stage	Source model	Format
llm/qwen3-1.7b-w8a8-rk3588.rkllm	LLM escalation tier	Qwen/Qwen3-1.7B	RKLLM w8a8, 3 NPU cores

Converted with rkllm-toolkit 1.3.0 via the repo's scripts/convert_rkllm.py.

Usage (on the board)

bash scripts/download_models.sh llm_rkllm
uv run intent-model --serve --llm-runtime rkllm \
    --asr-provider rknn --asr-model-dir models/asr-sensevoice-rknn

Requirements: RKNPU driver >= 0.9.8, librkllmrt.so. Qwen3-1.7B keeps thinking mode (enable_thinking); ~13 tok/s on RK3588.

Why no GLiNER2 encoder here

Converting the pipeline's GLiNER2 (DeBERTa-v3) encoder to RKNN was attempted thoroughly (fp16 + int8, graph splitting, every relevant toolkit knob) and is blocked by rknn-toolkit2 2.3.2 itself: its build phase needs >460 GB RAM at the model's working sequence length, crashes with integer-overflow mallocs, and emits artifacts with multi-GB zero-padded sections. On the board the GLiNER2 encoder runs on CPU via ONNX (--gliner-encoder onnx), which matches torch accuracy; the NPU's three cores are used by ASR + the LLM.