---
license: apache-2.0
base_model: Qwen/Qwen2.5-Coder-7B-Instruct
tags:
  - rkllm
  - rk3576
  - npu
  - qwen2.5-coder
  - edge-ai
---

# Qwen2.5-Coder-7B-Instruct — RK3576 (RKLLM)

Conversion of [Qwen/Qwen2.5-Coder-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct) to RKLLM format for **RK3576** NPU (NanoPi R76S, Orange Pi RK3576, and similar boards).

## Conversion details

- **Toolkit:** rkllm-toolkit 1.2.3
- **Compatible runtime:** rkllm-runtime >= 1.2.3
- **Quantization:** w4a16
- **NPU cores:** 2 (dual-core)
- **Max context:** 4096 tokens
- **Platform:** RK3576 (NOT compatible with RK3588)

## Usage with RKLLama

Place the `.rkllm` file in your models folder and create a `Modelfile` with the following content:

FROM="Qwen2.5-Coder-7B-Instruct-rk3576.rkllm"
HUGGINGFACE_PATH="Qwen/Qwen2.5-Coder-7B-Instruct"
TOKENIZER="/models/tokenizers/Qwen2.5-Coder-7B"

## Performance (NanoPi R76S, 16GB RAM)

- ~10-15 tok/s on dual-core NPU

## Important note

This model was compiled specifically for RK3576. If you're using an RK3588 (RK3588S, RK3588J), look for a different conversion — they are not binary compatible with each other.