Hugging Face's logo

inarikami
/
DeepSeek-R1-Distill-Qwen-32B-AWQ

Text Generation
Safetensors
qwen2
code
deepseek_v3
qwen
int4
conversational
4-bit precision
awq
Model card Files
xet
 Community
inarikami commited on
Commit
26b3b99
·
verified ·
1 Parent(s): 6ea4884

fix table, update quant source

Browse files
Files changed (1) hide show
  1. README.md +27 -28
README.md CHANGED
@@ -7,6 +7,8 @@ tags:
7
  - qwen
8
  - int4
9
  - conversational
 
 
10
  ---
11
 
12
  # DeepSeek-R1-Distill-Qwen-32B-AWQ wint4
@@ -25,31 +27,28 @@ The MMLU-PRO dataset evaluates subjects across 14 distinct fields using a 5-shot
25
 
26
  - **Shots**: 5-shot
27
 
28
- ### Results Table
29
-
30
- | Tasks | Version | Filter | n-shot | Metric | Direction | Value | Stderr |
31
- |---------------------------|---------|---------------|--------|------------|-----------|-------|--------|
32
- | mmlu_pro | 2 | custom-extract| | exact_match| ↑ | 0.5875| 0.0044 |
33
- | biology | 1 | custom-extract| 5 | exact_match| ↑ | 0.7978| 0.0150 |
34
- | business | 1 | custom-extract| 5 | exact_match| ↑ | 0.5982| 0.0175 |
35
- | chemistry | 1 | custom-extract| 5 | exact_match| ↑ | 0.4691| 0.0148 |
36
- | computer_science | 1 | custom-extract| 5 | exact_match| ↑ | 0.6122| 0.0241 |
37
- | economics | 1 | custom-extract| 5 | exact_match| ↑ | 0.7346| 0.0152 |
38
- | engineering | 1 | custom-extract| 5 | exact_match| ↑ | 0.3891| 0.0157 |
39
- | health | 1 | custom-extract| 5 | exact_match| ↑ | 0.6345| 0.0168 |
40
- | history | 1 | custom-extract| 5 | exact_match| ↑ | 0.6168| 0.0249 |
41
- | law | 1 | custom-extract| 5 | exact_match| ↑ | 0.4596| 0.0150 |
42
- | math | 1 | custom-extract| 5 | exact_match| ↑ | 0.6425| 0.0130 |
43
- | other | 1 | custom-extract| 5 | exact_match| ↑ | 0.6223| 0.0160 |
44
- | philosophy | 1 | custom-extract| 5 | exact_match| ↑ | 0.5731| 0.0222 |
45
- | physics | 1 | custom-extract| 5 | exact_match| ↑ | 0.5073| 0.0139 |
46
- | psychology | 1 | custom-extract| 5 | exact_match| ↑ | 0.7494| 0.0154 |
47
-
48
- ## Groups
49
-
50
- | Groups | Version | Filter | n-shot | Metric | Direction | Value | Stderr |
51
- |-----------|---------|---------------|--------|------------|-----------|-------|--------|
52
- | mmlu_pro | 2 | custom-extract| | exact_match| ↑ | 0.5875| 0.0044 |
53
-
54
-
55
-
 
7
  - qwen
8
  - int4
9
  - conversational
10
+ base_model:
11
+ - deepseek-ai/DeepSeek-R1-Distill-Qwen-32B
12
  ---
13
 
14
  # DeepSeek-R1-Distill-Qwen-32B-AWQ wint4
 
27
 
28
  - **Shots**: 5-shot
29
 
30
+ ### Tasks
31
+
32
+ | Tasks | Filter | n-shot | Metric | Value | Stderr |
33
+ |------------------|---------------|--------|-------------|--------|--------|
34
+ | mmlu_pro | custom-extract| | exact_match | 0.5875 | 0.0044 |
35
+ | biology | custom-extract| 5 | exact_match | 0.7978 | 0.0150 |
36
+ | business | custom-extract| 5 | exact_match | 0.5982 | 0.0175 |
37
+ | chemistry | custom-extract| 5 | exact_match | 0.4691 | 0.0148 |
38
+ | computer_science | custom-extract| 5 | exact_match | 0.6122 | 0.0241 |
39
+ | economics | custom-extract| 5 | exact_match | 0.7346 | 0.0152 |
40
+ | engineering | custom-extract| 5 | exact_match | 0.3891 | 0.0157 |
41
+ | health | custom-extract| 5 | exact_match | 0.6345 | 0.0168 |
42
+ | history | custom-extract| 5 | exact_match | 0.6168 | 0.0249 |
43
+ | law | custom-extract| 5 | exact_match | 0.4596 | 0.0150 |
44
+ | math | custom-extract| 5 | exact_match | 0.6425 | 0.0130 |
45
+ | other | custom-extract| 5 | exact_match | 0.6223 | 0.0160 |
46
+ | philosophy | custom-extract| 5 | exact_match | 0.5731 | 0.0222 |
47
+ | physics | custom-extract| 5 | exact_match | 0.5073 | 0.0139 |
48
+ | psychology | custom-extract| 5 | exact_match | 0.7494 | 0.0154 |
49
+
50
+ ### Groups
51
+
52
+ | Groups | Filter | n-shot | Metric | Value | Stderr |
53
+ |----------|---------------|--------|-------------|--------|--------|
54
+ | mmlu_pro | custom-extract| | exact_match | 0.5875 | 0.0044 |