HanzoHuang commited on
Commit
9460f29
·
verified ·
1 Parent(s): adf1e2d

Document RKLLM artifacts and checksums

Browse files
Files changed (1) hide show
  1. README.md +21 -22
README.md CHANGED
@@ -1,6 +1,7 @@
1
  ---
2
  license: apache-2.0
3
  base_model: google/gemma-4-E2B-it
 
4
  library_name: rkllm
5
  tags:
6
  - rkllm
@@ -13,32 +14,32 @@ tags:
13
 
14
  # gemma-4-E2B-it-RKLLM
15
 
16
- RKLLM-converted versions of Gemma 4 E2B IT for deployment on Rockchip RK3576 and RK3588 NPUs.
17
 
18
- `.rkllm` files require the Rockchip RKLLM runtime and cannot be loaded directly with Transformers, llama.cpp, or Ollama.
19
 
20
- ## Base Model
21
 
22
- - Model: Gemma 4 E2B IT
23
- - Author: Google
24
- - Original model: [google/gemma-4-E2B-it](https://huggingface.co/google/gemma-4-E2B-it)
25
- - Original license: Apache-2.0
26
 
27
- Refer to the upstream model card for the original model's capabilities, limitations, and acceptable-use guidance.
 
 
 
28
 
29
- ## Available Variants
30
 
31
- | Target | File | Quantization |
32
- | --- | --- | --- |
33
- | RK3576 | [`gemma-4-E2B-it_RK3576_w4a16_g128.rkllm`](RK3576/gemma-4-E2B-it_RK3576_w4a16_g128.rkllm) | W4A16 (g128) |
34
- | RK3576 | [`gemma-4-E2B-it_RK3576_w8a8.rkllm`](RK3576/gemma-4-E2B-it_RK3576_w8a8.rkllm) | W8A8 |
35
- | RK3588 | [`gemma-4-E2B-it_RK3588_w8a8.rkllm`](RK3588/gemma-4-E2B-it_RK3588_w8a8.rkllm) | W8A8 |
36
 
37
- The repository also contains `gemma-4-E2B-it_data_quant.json`, the calibration data used during conversion.
 
 
 
 
38
 
39
- ## Usage
40
 
41
- Download the model variant that matches your target SoC:
42
 
43
  ```bash
44
  hf download HanzoHuang/gemma-4-E2B-it-RKLLM \
@@ -46,14 +47,12 @@ hf download HanzoHuang/gemma-4-E2B-it-RKLLM \
46
  --local-dir gemma-4-E2B-it-RKLLM
47
  ```
48
 
49
- Load the `.rkllm` file with a compatible RKLLM runtime. Use the correct prompt formatting for the instruction-tuned upstream model.
50
 
51
  ## Limitations
52
 
53
- - These are hardware-specific converted artifacts, not Transformers checkpoints.
54
- - Runtime, driver, and toolkit compatibility depends on the Rockchip software stack installed on the device.
55
- - Conversion may change output quality relative to the upstream floating-point model; validate on your own workload.
56
 
57
  ## Acknowledgements
58
 
59
- Thanks to Google for releasing Gemma 4 E2B IT and to Rockchip and RKLLM contributors for the deployment toolchain.
 
1
  ---
2
  license: apache-2.0
3
  base_model: google/gemma-4-E2B-it
4
+ pipeline_tag: text-generation
5
  library_name: rkllm
6
  tags:
7
  - rkllm
 
14
 
15
  # gemma-4-E2B-it-RKLLM
16
 
17
+ RKLLM-converted Gemma 4 E2B IT language-model artifacts for Rockchip RK3576 and RK3588 NPUs.
18
 
19
+ Important: the upstream Gemma 4 E2B IT model is multimodal/VLM-capable, but this repository contains only the RKLLM language-model binaries. The converted model exposed here is LLM-only: it does not accept images and does not include an RKNN vision encoder.
20
 
21
+ These hardware-specific `.rkllm` files require a compatible Rockchip RKLLM runtime. They are not Transformers checkpoints and cannot be loaded directly with Transformers, llama.cpp, or Ollama.
22
 
23
+ ## Base model
 
 
 
24
 
25
+ - Upstream model: [google/gemma-4-E2B-it](https://huggingface.co/google/gemma-4-E2B-it)
26
+ - License: Apache-2.0
27
+ - Upstream type: VLM/multimodal
28
+ - RKLLM type: LLM-only
29
 
30
+ ## Conversion and variants
31
 
32
+ Converted with RKLLM Toolkit 1.3.0. Use a file built for the exact target SoC.
 
 
 
 
33
 
34
+ | Target | Quantization | File | SHA256 |
35
+ | --- | --- | --- | --- |
36
+ | RK3576 | W4A16 (g128) | [gemma-4-E2B-it_RK3576_w4a16_g128.rkllm](RK3576/gemma-4-E2B-it_RK3576_w4a16_g128.rkllm) | `973008fd7ebed81e2a26dcf7dcc1c28fde631837e3e021c7f3007d6e82647165` |
37
+ | RK3576 | W8A8 | [gemma-4-E2B-it_RK3576_w8a8.rkllm](RK3576/gemma-4-E2B-it_RK3576_w8a8.rkllm) | `b7357c01fcd51c8f896f06ee3a3746bc233031abd04e2fa0cebb5908b1382a01` |
38
+ | RK3588 | W8A8 | [gemma-4-E2B-it_RK3588_w8a8.rkllm](RK3588/gemma-4-E2B-it_RK3588_w8a8.rkllm) | `ae81d1e37eb54b40d2d366ef622a0c05acfb15537272e982efc1dfdd60557979` |
39
 
40
+ The repository also includes `gemma-4-E2B-it_data_quant.json`, used as calibration data during conversion.
41
 
42
+ ## Usage
43
 
44
  ```bash
45
  hf download HanzoHuang/gemma-4-E2B-it-RKLLM \
 
47
  --local-dir gemma-4-E2B-it-RKLLM
48
  ```
49
 
50
+ Run the file with the RKLLM runtime and the upstream Gemma instruction prompt format. For Docker deployment, see [Hanzo-Huang/rkllm-docker](https://github.com/Hanzo-Huang/rkllm-docker).
51
 
52
  ## Limitations
53
 
54
+ Vision input is not supported by these artifacts. Conversion can change output quality relative to the upstream model; validate the result on your target device.
 
 
55
 
56
  ## Acknowledgements
57
 
58
+ Thanks to Google, Rockchip, and the RKLLM community.