Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,83 @@
+---
+license: apache-2.0
+base_model: Qwen/Qwen2.5-1.5B-Instruct
+tags:
+- rkllm
+- rk3588
+- npu
+- quantized
+- w8a8
+pipeline_tag: text-generation
+---
+
+# Qwen2.5-1.5B-Instruct for RKLLM v1.2.3 (RK3588 NPU)
+
+Pre-converted Qwen2.5-1.5B-Instruct for the Rockchip RK3588 NPU using rknn-llm runtime v1.2.3.
+
+Runs on Orange Pi 5 Plus, Rock 5B, Radxa NX5, and other RK3588-based SBCs with 8GB+ RAM.
+
+## Files
+
+| File | Size | Description |
+|------|------|-------------|
+| `Qwen2.5-1.5B-Instruct-rk3588-v123-w8a8.rkllm` | 2.0 GB | LLM (W8A8 quantized, 8192 token context) |
+
+## Quick Start
+
+### Directory Structure
+
+```
+~/models/Qwen2.5-1.5B-Instruct/
+    Qwen2.5-1.5B-Instruct-rk3588-v123-w8a8.rkllm
+```
+
+### Using with RKLLM API Server
+
+This model is designed for use with the [RKLLM API Server](https://github.com/airockchip/rknn-llm), which provides an OpenAI-compatible API for RK3588 NPU inference. The server auto-discovers `.rkllm` files by scanning subdirectories of your models folder.
+
+```bash
+# Place the model in your models directory
+mkdir -p ~/models/Qwen2.5-1.5B-Instruct
+# Copy .rkllm file here — the API server will find it automatically
+sudo systemctl restart rkllm-api
+```
+
+The model will appear as `qwen2.5-1.5b-instruct` in the OpenAI-compatible model list.
+
+## Export Details
+
+| Parameter | Value |
+|-----------|-------|
+| Source | [Qwen/Qwen2.5-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct) |
+| Tool | rkllm-toolkit v1.2.3 |
+| Quantization | W8A8 (8-bit weights, 8-bit activations) |
+| Optimization level | 1 |
+| Target platform | rk3588 |
+| NPU cores | 3 |
+| Max context | 8192 tokens |
+
+## Performance Benchmarks
+
+Tested on Orange Pi 5 Plus (16GB RAM), RK3588 SoC, RKNPU driver 0.9.8:
+
+| Metric | Value |
+|--------|-------|
+| Decode speed | ~19 tok/s |
+| Model load time | ~3 s |
+| Peak RAM | ~2.2 GB |
+
+## Known Limitations
+
+- **Tool calling**: Qwen2.5-1.5B is a general-purpose instruct model, not fine-tuned for structured function calling. For agent/tool-calling workloads, see [GatekeeperZA/xLAM-1b-fc-r-RKLLM-v1.2.3](https://huggingface.co/GatekeeperZA/xLAM-1b-fc-r-RKLLM-v1.2.3).
+- **Context**: Compiled at 8192 tokens. The runtime will truncate inputs that exceed this limit.
+- **Version lock**: This file requires RKLLM runtime v1.2.3 and NPU driver v0.9.8. It is **not compatible** with v1.3.0 or later runtimes.
+
+## License
+
+Apache 2.0, inherited from Qwen2.5-1.5B-Instruct.
+
+## Credits
+
+- Model: [Qwen Team](https://huggingface.co/Qwen) for Qwen2.5-1.5B-Instruct
+- Runtime: [Rockchip / airockchip](https://github.com/airockchip/rknn-llm) for rkllm-toolkit
+- API Server: [RKLLM API Server](https://github.com/airockchip/rknn-llm) — OpenAI-compatible server for RK3588 NPU