Add model card
Browse files
README.md
ADDED
|
@@ -0,0 +1,81 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
base_model: OpenGVLab/InternVL3-4B
|
| 4 |
+
tags:
|
| 5 |
+
- rkllm
|
| 6 |
+
- rknn
|
| 7 |
+
- rk3588
|
| 8 |
+
- rockchip
|
| 9 |
+
- npu
|
| 10 |
+
- quantized
|
| 11 |
+
- vision-language
|
| 12 |
+
- multimodal
|
| 13 |
+
language:
|
| 14 |
+
- en
|
| 15 |
+
- zh
|
| 16 |
+
pipeline_tag: image-text-to-text
|
| 17 |
+
---
|
| 18 |
+
|
| 19 |
+
# InternVL3.5-4B-Instruct — RKLLM v1.2.3 (w8a8, RK3588)
|
| 20 |
+
|
| 21 |
+
RKLLM/RKNN conversion of [OpenGVLab/InternVL3-4B](https://huggingface.co/OpenGVLab/InternVL3-4B) for Rockchip RK3588 NPU inference.
|
| 22 |
+
|
| 23 |
+
Converted with RKLLM Toolkit v1.2.3 (language model) and RKNN Toolkit (vision encoder). This is a multimodal vision-language model — it accepts both images and text as input.
|
| 24 |
+
|
| 25 |
+
## Key Details
|
| 26 |
+
|
| 27 |
+
| Property | Value |
|
| 28 |
+
|----------|-------|
|
| 29 |
+
| Base Model | OpenGVLab/InternVL3-4B |
|
| 30 |
+
| Toolkit Version | RKLLM Toolkit v1.2.3 / RKNN Toolkit |
|
| 31 |
+
| Runtime Version | RKLLM Runtime ≥ v1.2.1 + RKNN Runtime |
|
| 32 |
+
| Quantization | w8a8 (8-bit weights, 8-bit activations) |
|
| 33 |
+
| Target Platform | RK3588 |
|
| 34 |
+
| NPU Cores | 3 |
|
| 35 |
+
| Thinking Mode | ❌ Not applicable |
|
| 36 |
+
| Model Type | Vision-Language (VLM) |
|
| 37 |
+
| Languages | English, Chinese (multilingual) |
|
| 38 |
+
|
| 39 |
+
## Why This Model?
|
| 40 |
+
|
| 41 |
+
InternVL3.5-4B is Shanghai AI Lab's compact vision-language model. It provides image understanding, visual question answering, and OCR capabilities at 4B parameters — all running on the RK3588 NPU without a GPU.
|
| 42 |
+
|
| 43 |
+
Compared to the Qwen3-VL-4B, InternVL has a different training lineage and excels at dense image analysis and chart/document understanding.
|
| 44 |
+
|
| 45 |
+
## Hardware Tested
|
| 46 |
+
|
| 47 |
+
- **Orange Pi 5 Plus** — RK3588, 16GB RAM, Armbian Linux
|
| 48 |
+
- RKNPU driver 0.9.8
|
| 49 |
+
- RKLLM Runtime v1.2.3
|
| 50 |
+
|
| 51 |
+
## Usage
|
| 52 |
+
|
| 53 |
+
### With the RKLLM API Server (VLM mode)
|
| 54 |
+
|
| 55 |
+
```bash
|
| 56 |
+
mkdir -p ~/models/internvl3.5-4b
|
| 57 |
+
cd ~/models/internvl3.5-4b
|
| 58 |
+
git lfs install && git clone https://huggingface.co/GatekeeperZA/InternVL3.5-4B-Instruct-RKLLM-v1.2.3 .
|
| 59 |
+
```
|
| 60 |
+
|
| 61 |
+
Use with [GatekeeperZA/RKLLM-API-Server](https://github.com/GatekeeperZA/RKLLM-API-Server) — the server loads both the `.rkllm` and `.rknn` files automatically when placed in the same directory.
|
| 62 |
+
|
| 63 |
+
## File Listing
|
| 64 |
+
|
| 65 |
+
| File | Description |
|
| 66 |
+
|------|-------------|
|
| 67 |
+
| `internvl3_5-4b-instruct_w8a8_rk3588.rkllm` | Language model weights for RK3588 NPU |
|
| 68 |
+
| `internvl3_5-4b_vision_rk3588.rknn` | Vision encoder for RK3588 NPU |
|
| 69 |
+
|
| 70 |
+
## Compatibility Notes
|
| 71 |
+
|
| 72 |
+
- Minimum runtime: RKLLM Runtime v1.2.1 + RKNN Runtime v2.x. v1.2.3 recommended.
|
| 73 |
+
- RKNPU driver: ≥ 0.9.6
|
| 74 |
+
- SoCs: RK3588 / RK3588S. Not compatible with RK3576 without reconversion.
|
| 75 |
+
- RAM: ~5.5GB loaded. Requires 8GB+ board (16GB recommended).
|
| 76 |
+
|
| 77 |
+
## Acknowledgements
|
| 78 |
+
|
| 79 |
+
- Shanghai AI Lab / OpenGVLab for InternVL3
|
| 80 |
+
- Rockchip / airockchip for the RKLLM and RKNN toolkits
|
| 81 |
+
- Converted by [GatekeeperZA](https://huggingface.co/GatekeeperZA)
|