Improve language tag

#2
by lbourdois - opened
Files changed (1) hide show
  1. README.md +166 -152
README.md CHANGED
@@ -1,152 +1,166 @@
1
- ---
2
- library_name: transformers
3
- tags:
4
- - mergekit
5
- - merge
6
- base_model:
7
- - Qwen/Qwen2.5-7B
8
- - Qwen/Qwen2.5-7B-Instruct
9
- infernce: true
10
- model-index:
11
- - name: model-3
12
- results:
13
- - task:
14
- type: text-generation
15
- name: Text Generation
16
- dataset:
17
- name: IFEval (0-Shot)
18
- type: HuggingFaceH4/ifeval
19
- args:
20
- num_few_shot: 0
21
- metrics:
22
- - type: inst_level_strict_acc and prompt_level_strict_acc
23
- value: 62.64
24
- name: strict accuracy
25
- source:
26
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Sakalti/model-3
27
- name: Open LLM Leaderboard
28
- - task:
29
- type: text-generation
30
- name: Text Generation
31
- dataset:
32
- name: BBH (3-Shot)
33
- type: BBH
34
- args:
35
- num_few_shot: 3
36
- metrics:
37
- - type: acc_norm
38
- value: 36.32
39
- name: normalized accuracy
40
- source:
41
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Sakalti/model-3
42
- name: Open LLM Leaderboard
43
- - task:
44
- type: text-generation
45
- name: Text Generation
46
- dataset:
47
- name: MATH Lvl 5 (4-Shot)
48
- type: hendrycks/competition_math
49
- args:
50
- num_few_shot: 4
51
- metrics:
52
- - type: exact_match
53
- value: 32.33
54
- name: exact match
55
- source:
56
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Sakalti/model-3
57
- name: Open LLM Leaderboard
58
- - task:
59
- type: text-generation
60
- name: Text Generation
61
- dataset:
62
- name: GPQA (0-shot)
63
- type: Idavidrein/gpqa
64
- args:
65
- num_few_shot: 0
66
- metrics:
67
- - type: acc_norm
68
- value: 9.51
69
- name: acc_norm
70
- source:
71
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Sakalti/model-3
72
- name: Open LLM Leaderboard
73
- - task:
74
- type: text-generation
75
- name: Text Generation
76
- dataset:
77
- name: MuSR (0-shot)
78
- type: TAUR-Lab/MuSR
79
- args:
80
- num_few_shot: 0
81
- metrics:
82
- - type: acc_norm
83
- value: 11.5
84
- name: acc_norm
85
- source:
86
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Sakalti/model-3
87
- name: Open LLM Leaderboard
88
- - task:
89
- type: text-generation
90
- name: Text Generation
91
- dataset:
92
- name: MMLU-PRO (5-shot)
93
- type: TIGER-Lab/MMLU-Pro
94
- config: main
95
- split: test
96
- args:
97
- num_few_shot: 5
98
- metrics:
99
- - type: acc
100
- value: 38.39
101
- name: accuracy
102
- source:
103
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Sakalti/model-3
104
- name: Open LLM Leaderboard
105
- ---
106
- # merge
107
-
108
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
109
-
110
- ## Merge Details
111
- ### Merge Method
112
-
113
- This model was merged using the [TIES](https://arxiv.org/abs/2306.01708) merge method using [Qwen/Qwen2.5-7B](https://huggingface.co/Qwen/Qwen2.5-7B) as a base.
114
-
115
- ### Models Merged
116
-
117
- The following models were included in the merge:
118
- * [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct)
119
-
120
- ### Configuration
121
-
122
- The following YAML configuration was used to produce this model:
123
-
124
- ```yaml
125
- models:
126
- - model: Qwen/Qwen2.5-7B-Instruct
127
- parameters:
128
- weight: 1
129
- density: 1
130
- merge_method: ties
131
- base_model: Qwen/Qwen2.5-7B
132
- parameters:
133
- weight: 1
134
- density: 1
135
- normalize: true
136
- int8_mask: true
137
- dtype: bfloat16
138
- ```
139
-
140
- # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
141
- Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/Sakalti__model-3-details)
142
-
143
- | Metric |Value|
144
- |-------------------|----:|
145
- |Avg. |31.78|
146
- |IFEval (0-Shot) |62.64|
147
- |BBH (3-Shot) |36.32|
148
- |MATH Lvl 5 (4-Shot)|32.33|
149
- |GPQA (0-shot) | 9.51|
150
- |MuSR (0-shot) |11.50|
151
- |MMLU-PRO (5-shot) |38.39|
152
-
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: transformers
3
+ tags:
4
+ - mergekit
5
+ - merge
6
+ base_model:
7
+ - Qwen/Qwen2.5-7B
8
+ - Qwen/Qwen2.5-7B-Instruct
9
+ infernce: true
10
+ language:
11
+ - zho
12
+ - eng
13
+ - fra
14
+ - spa
15
+ - por
16
+ - deu
17
+ - ita
18
+ - rus
19
+ - jpn
20
+ - kor
21
+ - vie
22
+ - tha
23
+ - ara
24
+ model-index:
25
+ - name: model-3
26
+ results:
27
+ - task:
28
+ type: text-generation
29
+ name: Text Generation
30
+ dataset:
31
+ name: IFEval (0-Shot)
32
+ type: HuggingFaceH4/ifeval
33
+ args:
34
+ num_few_shot: 0
35
+ metrics:
36
+ - type: inst_level_strict_acc and prompt_level_strict_acc
37
+ value: 62.64
38
+ name: strict accuracy
39
+ source:
40
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Sakalti/model-3
41
+ name: Open LLM Leaderboard
42
+ - task:
43
+ type: text-generation
44
+ name: Text Generation
45
+ dataset:
46
+ name: BBH (3-Shot)
47
+ type: BBH
48
+ args:
49
+ num_few_shot: 3
50
+ metrics:
51
+ - type: acc_norm
52
+ value: 36.32
53
+ name: normalized accuracy
54
+ source:
55
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Sakalti/model-3
56
+ name: Open LLM Leaderboard
57
+ - task:
58
+ type: text-generation
59
+ name: Text Generation
60
+ dataset:
61
+ name: MATH Lvl 5 (4-Shot)
62
+ type: hendrycks/competition_math
63
+ args:
64
+ num_few_shot: 4
65
+ metrics:
66
+ - type: exact_match
67
+ value: 32.33
68
+ name: exact match
69
+ source:
70
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Sakalti/model-3
71
+ name: Open LLM Leaderboard
72
+ - task:
73
+ type: text-generation
74
+ name: Text Generation
75
+ dataset:
76
+ name: GPQA (0-shot)
77
+ type: Idavidrein/gpqa
78
+ args:
79
+ num_few_shot: 0
80
+ metrics:
81
+ - type: acc_norm
82
+ value: 9.51
83
+ name: acc_norm
84
+ source:
85
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Sakalti/model-3
86
+ name: Open LLM Leaderboard
87
+ - task:
88
+ type: text-generation
89
+ name: Text Generation
90
+ dataset:
91
+ name: MuSR (0-shot)
92
+ type: TAUR-Lab/MuSR
93
+ args:
94
+ num_few_shot: 0
95
+ metrics:
96
+ - type: acc_norm
97
+ value: 11.5
98
+ name: acc_norm
99
+ source:
100
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Sakalti/model-3
101
+ name: Open LLM Leaderboard
102
+ - task:
103
+ type: text-generation
104
+ name: Text Generation
105
+ dataset:
106
+ name: MMLU-PRO (5-shot)
107
+ type: TIGER-Lab/MMLU-Pro
108
+ config: main
109
+ split: test
110
+ args:
111
+ num_few_shot: 5
112
+ metrics:
113
+ - type: acc
114
+ value: 38.39
115
+ name: accuracy
116
+ source:
117
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Sakalti/model-3
118
+ name: Open LLM Leaderboard
119
+ ---
120
+ # merge
121
+
122
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
123
+
124
+ ## Merge Details
125
+ ### Merge Method
126
+
127
+ This model was merged using the [TIES](https://arxiv.org/abs/2306.01708) merge method using [Qwen/Qwen2.5-7B](https://huggingface.co/Qwen/Qwen2.5-7B) as a base.
128
+
129
+ ### Models Merged
130
+
131
+ The following models were included in the merge:
132
+ * [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct)
133
+
134
+ ### Configuration
135
+
136
+ The following YAML configuration was used to produce this model:
137
+
138
+ ```yaml
139
+ models:
140
+ - model: Qwen/Qwen2.5-7B-Instruct
141
+ parameters:
142
+ weight: 1
143
+ density: 1
144
+ merge_method: ties
145
+ base_model: Qwen/Qwen2.5-7B
146
+ parameters:
147
+ weight: 1
148
+ density: 1
149
+ normalize: true
150
+ int8_mask: true
151
+ dtype: bfloat16
152
+ ```
153
+
154
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
155
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/Sakalti__model-3-details)
156
+
157
+ | Metric |Value|
158
+ |-------------------|----:|
159
+ |Avg. |31.78|
160
+ |IFEval (0-Shot) |62.64|
161
+ |BBH (3-Shot) |36.32|
162
+ |MATH Lvl 5 (4-Shot)|32.33|
163
+ |GPQA (0-shot) | 9.51|
164
+ |MuSR (0-shot) |11.50|
165
+ |MMLU-PRO (5-shot) |38.39|
166
+