Add model card

README.md

@@ -0,0 +1,112 @@
+---
+license: llama3.2
+base_model: meta-llama/Llama-3.2-3B-Instruct
+tags:
+  - rkllm
+  - rk3588
+  - rockchip
+  - npu
+  - quantized
+  - llama
+language:
+  - en
+---
+
+# Llama-3.2-3B-Instruct — RKLLM v1.2.3 (w8a8, RK3588)
+
+RKLLM conversion of [meta-llama/Llama-3.2-3B-Instruct](https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct) for Rockchip RK3588 NPU inference.
+
+Converted with RKLLM Toolkit v1.2.3. This is a standard instruct model — it does **not** produce `<think>` reasoning blocks.
+
+## Key Details
+
+| Property | Value |
+|----------|-------|
+| Base Model | meta-llama/Llama-3.2-3B-Instruct |
+| Toolkit Version | RKLLM Toolkit v1.2.3 |
+| Runtime Version | RKLLM Runtime ≥ v1.2.1 (v1.2.3 recommended) |
+| Quantization | w8a8 (8-bit weights, 8-bit activations) |
+| Quantization Algorithm | normal |
+| Target Platform | RK3588 |
+| NPU Cores | 3 |
+| Max Context Length | 8192 tokens |
+| Optimization Level | 0 |
+| Thinking Mode | ❌ Not supported |
+| Languages | English (+ multilingual inherited from Llama 3.2) |
+
+## Why This Model?
+
+Llama 3.2 3B Instruct is Meta's latest compact instruction model. It brings a different architecture and training lineage to the RK3588 NPU lineup — strong at instruction following, coding, and general reasoning without the overhead of a thinking/reasoning mode.
+
+At ~3B parameters it sits between the 1.7B and 4B Qwen3 models, offering a useful middle ground.
+
+## Hardware Tested
+
+- **Orange Pi 5 Plus** — RK3588, 16GB RAM, Armbian Linux
+- RKNPU driver 0.9.8
+- RKLLM Runtime v1.2.3
+
+## Usage
+
+### With the official RKLLM API demo
+
+```bash
+# Clone the runtime
+git clone https://github.com/airockchip/rknn-llm.git
+cd rknn-llm/examples/rkllm_api_demo
+
+# Run (aarch64)
+./build/rkllm_api_demo /path/to/Llama-3.2-3B-Instruct-rk3588-w8a8.rkllm 4096 8192
+```
+
+### With the RKLLM API Server
+
+Download and place in `~/models/`:
+
+```bash
+mkdir -p ~/models/Llama-3.2-3B-Instruct
+cd ~/models/Llama-3.2-3B-Instruct
+git lfs install && git clone https://huggingface.co/GatekeeperZA/Llama-3.2-3B-Instruct-RKLLM-v1.2.3 .
+```
+
+The server auto-detects the model as `instruct` capability (no thinking). Use with [GatekeeperZA/RKLLM-API-Server](https://github.com/GatekeeperZA/RKLLM-API-Server).
+
+## Conversion Script
+
+```python
+from rkllm.api import RKLLM
+
+llm = RKLLM()
+llm.load_huggingface(model="meta-llama/Llama-3.2-3B-Instruct", device="cpu", dtype="float32")
+llm.build(
+    do_quantization=True,
+    optimization_level=0,
+    quantized_dtype="w8a8",
+    quantized_algorithm="normal",
+    target_platform="rk3588",
+    num_npu_core=3,
+    max_context=8192,
+)
+llm.export_rkllm("./Llama-3.2-3B-Instruct-rk3588-w8a8.rkllm")
+```
+
+> **WSL2 note:** Requires ≥16GB WSL2 memory (`memory=16GB` in `~/.wslconfig`). Write the output to a native Linux path (`/home/user/`) first, then copy to `/mnt/` — writing directly to the Windows mount triggers OOM during the export phase.
+
+## File Listing
+
+| File | Description |
+|------|-------------|
+| `Llama-3.2-3B-Instruct-rk3588-w8a8.rkllm` | Quantized model for RK3588 NPU |
+
+## Compatibility Notes
+
+- Minimum runtime: RKLLM Runtime v1.2.1. v1.2.3 recommended.
+- RKNPU driver: ≥ 0.9.6
+- SoCs: RK3588 / RK3588S (3 NPU cores). Not compatible with RK3576 (2 cores) without reconversion.
+- RAM: ~3.5GB loaded. Runs comfortably on 8GB+ boards.
+
+## Acknowledgements
+
+- Meta / FAIR for the Llama 3.2 base model
+- Rockchip / airockchip for the RKLLM toolkit and runtime
+- Converted by [GatekeeperZA](https://huggingface.co/GatekeeperZA)