HanzoHuang commited on
Commit
1b3d534
·
verified ·
1 Parent(s): 5fc8fff

Complete RKLLM model card

Browse files
Files changed (1) hide show
  1. README.md +57 -0
README.md CHANGED
@@ -1,3 +1,60 @@
1
  ---
2
  license: mit
 
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: mit
3
+ base_model: deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B
4
+ pipeline_tag: text-generation
5
+ library_name: rkllm
6
+ tags:
7
+ - rkllm
8
+ - rockchip
9
+ - rk3576
10
+ - rk3588
11
+ - deepseek
12
+ - deepseek-r1
13
+ - qwen
14
  ---
15
+
16
+ # DeepSeek-R1-Distill-Qwen-1.5B-RKLLM
17
+
18
+ RKLLM-converted DeepSeek-R1-Distill-Qwen-1.5B language-model artifacts for Rockchip RK3576 and RK3588 NPUs.
19
+
20
+ These hardware-specific `.rkllm` files require a compatible Rockchip RKLLM runtime. They are not Transformers checkpoints and cannot be loaded directly with Transformers, llama.cpp, or Ollama.
21
+
22
+ ## Base model
23
+
24
+ - Upstream model: [deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B)
25
+ - License: MIT
26
+ - Model type: LLM (text only)
27
+
28
+ ## Conversion and variants
29
+
30
+ ### Toolkit version
31
+
32
+ **RKLLM Toolkit: v1.2.3**
33
+
34
+ Use a file built for the exact target SoC.
35
+
36
+ | Target | Quantization | File | SHA256 |
37
+ | --- | --- | --- | --- |
38
+ | RK3576 | W4A16-g128 | [DeepSeek-R1-Distill-Qwen-1.5B_RK3576_w4a16_g128.rkllm](RK3576/DeepSeek-R1-Distill-Qwen-1.5B_RK3576_w4a16_g128.rkllm) | `e737c9aa8cfbfd0216e547ed67bc2325d5add2f8b2aa21037389e296ac3b8d80` |
39
+ | RK3576 | W8A8 | [DeepSeek-R1-Distill-Qwen-1.5B_RK3576_w8a8.rkllm](RK3576/DeepSeek-R1-Distill-Qwen-1.5B_RK3576_w8a8.rkllm) | `6ceab2c6b93a8b55837ab75e4050bbce888030153c0bc83a465bd73ead44120f` |
40
+ | RK3588 | W8A8 | [DeepSeek-R1-Distill-Qwen-1.5B_RK3588_w8a8.rkllm](RK3588/DeepSeek-R1-Distill-Qwen-1.5B_RK3588_w8a8.rkllm) | `d00038d619c1905ac66ab5d499019d80889d37cc1f81be491cc503e284950c2d` |
41
+
42
+ The repository also includes `DeepSeek-R1-Distill-Qwen-1.5B_data_quant.json`, used as calibration data during conversion.
43
+
44
+ ## Usage
45
+
46
+ ```bash
47
+ hf download HanzoHuang/DeepSeek-R1-Distill-Qwen-1.5B-RKLLM \
48
+ RK3576/DeepSeek-R1-Distill-Qwen-1.5B_RK3576_w4a16_g128.rkllm \
49
+ --local-dir DeepSeek-R1-Distill-Qwen-1.5B-RKLLM
50
+ ```
51
+
52
+ Use the upstream DeepSeek R1 chat template with the RKLLM runtime. For Docker deployment, see [Hanzo-Huang/rkllm-docker](https://github.com/Hanzo-Huang/rkllm-docker).
53
+
54
+ ## Limitations
55
+
56
+ These are target-specific converted artifacts. Quantization and conversion can change output quality relative to the upstream model; validate the result on your Rockchip device.
57
+
58
+ ## Acknowledgements
59
+
60
+ Thanks to DeepSeek, the Qwen Team, Rockchip, and the RKLLM community.