Hugging Face's logo

prithivMLmods
/
Calcium-Opus-14B-Merge

Text Generation
Transformers
Safetensors
qwen2
mergekit
Merge
conversational
Eval Results (legacy)
text-generation-inference
Model card Files
xet
Community
2

Improve language tag

#2
by lbourdois - opened
base: refs/heads/main
←
from: refs/pr/2
Discussion Files changed
+170
-156
Files changed (1) hide show
  1. README.md +170 -156
README.md CHANGED
@@ -1,156 +1,170 @@
1
- ---
2
- base_model:
3
- - Qwen/Qwen2.5-14B-Instruct
4
- - prithivMLmods/Calcium-Opus-14B-Elite
5
- - prithivMLmods/QwQ-LCoT-14B-Conversational
6
- library_name: transformers
7
- tags:
8
- - mergekit
9
- - merge
10
- model-index:
11
- - name: Calcium-Opus-14B-Merge
12
- results:
13
- - task:
14
- type: text-generation
15
- name: Text Generation
16
- dataset:
17
- name: IFEval (0-Shot)
18
- type: wis-k/instruction-following-eval
19
- split: train
20
- args:
21
- num_few_shot: 0
22
- metrics:
23
- - type: inst_level_strict_acc and prompt_level_strict_acc
24
- value: 49.49
25
- name: averaged accuracy
26
- source:
27
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FCalcium-Opus-14B-Merge
28
- name: Open LLM Leaderboard
29
- - task:
30
- type: text-generation
31
- name: Text Generation
32
- dataset:
33
- name: BBH (3-Shot)
34
- type: SaylorTwift/bbh
35
- split: test
36
- args:
37
- num_few_shot: 3
38
- metrics:
39
- - type: acc_norm
40
- value: 46.77
41
- name: normalized accuracy
42
- source:
43
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FCalcium-Opus-14B-Merge
44
- name: Open LLM Leaderboard
45
- - task:
46
- type: text-generation
47
- name: Text Generation
48
- dataset:
49
- name: MATH Lvl 5 (4-Shot)
50
- type: lighteval/MATH-Hard
51
- split: test
52
- args:
53
- num_few_shot: 4
54
- metrics:
55
- - type: exact_match
56
- value: 33.08
57
- name: exact match
58
- source:
59
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FCalcium-Opus-14B-Merge
60
- name: Open LLM Leaderboard
61
- - task:
62
- type: text-generation
63
- name: Text Generation
64
- dataset:
65
- name: GPQA (0-shot)
66
- type: Idavidrein/gpqa
67
- split: train
68
- args:
69
- num_few_shot: 0
70
- metrics:
71
- - type: acc_norm
72
- value: 16.11
73
- name: acc_norm
74
- source:
75
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FCalcium-Opus-14B-Merge
76
- name: Open LLM Leaderboard
77
- - task:
78
- type: text-generation
79
- name: Text Generation
80
- dataset:
81
- name: MuSR (0-shot)
82
- type: TAUR-Lab/MuSR
83
- args:
84
- num_few_shot: 0
85
- metrics:
86
- - type: acc_norm
87
- value: 20.93
88
- name: acc_norm
89
- source:
90
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FCalcium-Opus-14B-Merge
91
- name: Open LLM Leaderboard
92
- - task:
93
- type: text-generation
94
- name: Text Generation
95
- dataset:
96
- name: MMLU-PRO (5-shot)
97
- type: TIGER-Lab/MMLU-Pro
98
- config: main
99
- split: test
100
- args:
101
- num_few_shot: 5
102
- metrics:
103
- - type: acc
104
- value: 48.4
105
- name: accuracy
106
- source:
107
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FCalcium-Opus-14B-Merge
108
- name: Open LLM Leaderboard
109
- ---
110
- # **Calcium-Opus-14B-Merge**
111
-
112
- Calcium-Opus-14B-Merge is based on the Qwen 2.5 14B modality architecture, designed to enhance the reasoning capabilities of 14B-parameter models. These models have proven effective in context understanding, reasoning, and mathematical problem-solving. It has been fine-tuned using a long chain-of-thought reasoning model and specialized datasets, with a focus on chain-of-thought (CoT) reasoning for problem-solving. This model is optimized for tasks requiring logical reasoning, detailed explanations, and multi-step problem-solving, making it ideal for applications such as instruction-following, text generation, and complex reasoning tasks.
113
-
114
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
115
-
116
- ### Merge Method
117
-
118
- This model was merged using the [Model Stock](https://arxiv.org/abs/2403.19522) merge method using [Qwen/Qwen2.5-14B-Instruct](https://huggingface.co/Qwen/Qwen2.5-14B-Instruct) as a base.
119
-
120
- ### Models Merged
121
-
122
- The following models were included in the merge:
123
- * [prithivMLmods/Calcium-Opus-14B-Elite](https://huggingface.co/prithivMLmods/Calcium-Opus-14B-Elite)
124
- * [prithivMLmods/QwQ-LCoT-14B-Conversational](https://huggingface.co/prithivMLmods/QwQ-LCoT-14B-Conversational)
125
-
126
- ### Configuration
127
-
128
- The following YAML configuration was used to produce this model:
129
-
130
- ```yaml
131
- models:
132
- - model: prithivMLmods/Calcium-Opus-14B-Elite
133
- - model: prithivMLmods/QwQ-LCoT-14B-Conversational
134
- merge_method: model_stock
135
- base_model: Qwen/Qwen2.5-14B-Instruct
136
- parameters:
137
- normalize: false
138
- int8_mask: true
139
- dtype: bfloat16
140
- tokenizer_source: "Qwen/Qwen2.5-14B-Instruct"
141
- ```
142
-
143
- # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
144
- Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/prithivMLmods__Calcium-Opus-14B-Merge-details)!
145
- Summarized results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/contents/viewer/default/train?q=prithivMLmods%2FCalcium-Opus-14B-Merge&sort[column]=Average%20%E2%AC%86%EF%B8%8F&sort[direction]=desc)!
146
-
147
- | Metric |Value (%)|
148
- |-------------------|--------:|
149
- |**Average** | 35.80|
150
- |IFEval (0-Shot) | 49.49|
151
- |BBH (3-Shot) | 46.77|
152
- |MATH Lvl 5 (4-Shot)| 33.08|
153
- |GPQA (0-shot) | 16.11|
154
- |MuSR (0-shot) | 20.93|
155
- |MMLU-PRO (5-shot) | 48.40|
156
-
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model:
3
+ - Qwen/Qwen2.5-14B-Instruct
4
+ - prithivMLmods/Calcium-Opus-14B-Elite
5
+ - prithivMLmods/QwQ-LCoT-14B-Conversational
6
+ library_name: transformers
7
+ tags:
8
+ - mergekit
9
+ - merge
10
+ language:
11
+ - zho
12
+ - eng
13
+ - fra
14
+ - spa
15
+ - por
16
+ - deu
17
+ - ita
18
+ - rus
19
+ - jpn
20
+ - kor
21
+ - vie
22
+ - tha
23
+ - ara
24
+ model-index:
25
+ - name: Calcium-Opus-14B-Merge
26
+ results:
27
+ - task:
28
+ type: text-generation
29
+ name: Text Generation
30
+ dataset:
31
+ name: IFEval (0-Shot)
32
+ type: wis-k/instruction-following-eval
33
+ split: train
34
+ args:
35
+ num_few_shot: 0
36
+ metrics:
37
+ - type: inst_level_strict_acc and prompt_level_strict_acc
38
+ value: 49.49
39
+ name: averaged accuracy
40
+ source:
41
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FCalcium-Opus-14B-Merge
42
+ name: Open LLM Leaderboard
43
+ - task:
44
+ type: text-generation
45
+ name: Text Generation
46
+ dataset:
47
+ name: BBH (3-Shot)
48
+ type: SaylorTwift/bbh
49
+ split: test
50
+ args:
51
+ num_few_shot: 3
52
+ metrics:
53
+ - type: acc_norm
54
+ value: 46.77
55
+ name: normalized accuracy
56
+ source:
57
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FCalcium-Opus-14B-Merge
58
+ name: Open LLM Leaderboard
59
+ - task:
60
+ type: text-generation
61
+ name: Text Generation
62
+ dataset:
63
+ name: MATH Lvl 5 (4-Shot)
64
+ type: lighteval/MATH-Hard
65
+ split: test
66
+ args:
67
+ num_few_shot: 4
68
+ metrics:
69
+ - type: exact_match
70
+ value: 33.08
71
+ name: exact match
72
+ source:
73
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FCalcium-Opus-14B-Merge
74
+ name: Open LLM Leaderboard
75
+ - task:
76
+ type: text-generation
77
+ name: Text Generation
78
+ dataset:
79
+ name: GPQA (0-shot)
80
+ type: Idavidrein/gpqa
81
+ split: train
82
+ args:
83
+ num_few_shot: 0
84
+ metrics:
85
+ - type: acc_norm
86
+ value: 16.11
87
+ name: acc_norm
88
+ source:
89
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FCalcium-Opus-14B-Merge
90
+ name: Open LLM Leaderboard
91
+ - task:
92
+ type: text-generation
93
+ name: Text Generation
94
+ dataset:
95
+ name: MuSR (0-shot)
96
+ type: TAUR-Lab/MuSR
97
+ args:
98
+ num_few_shot: 0
99
+ metrics:
100
+ - type: acc_norm
101
+ value: 20.93
102
+ name: acc_norm
103
+ source:
104
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FCalcium-Opus-14B-Merge
105
+ name: Open LLM Leaderboard
106
+ - task:
107
+ type: text-generation
108
+ name: Text Generation
109
+ dataset:
110
+ name: MMLU-PRO (5-shot)
111
+ type: TIGER-Lab/MMLU-Pro
112
+ config: main
113
+ split: test
114
+ args:
115
+ num_few_shot: 5
116
+ metrics:
117
+ - type: acc
118
+ value: 48.4
119
+ name: accuracy
120
+ source:
121
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FCalcium-Opus-14B-Merge
122
+ name: Open LLM Leaderboard
123
+ ---
124
+ # **Calcium-Opus-14B-Merge**
125
+
126
+ Calcium-Opus-14B-Merge is based on the Qwen 2.5 14B modality architecture, designed to enhance the reasoning capabilities of 14B-parameter models. These models have proven effective in context understanding, reasoning, and mathematical problem-solving. It has been fine-tuned using a long chain-of-thought reasoning model and specialized datasets, with a focus on chain-of-thought (CoT) reasoning for problem-solving. This model is optimized for tasks requiring logical reasoning, detailed explanations, and multi-step problem-solving, making it ideal for applications such as instruction-following, text generation, and complex reasoning tasks.
127
+
128
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
129
+
130
+ ### Merge Method
131
+
132
+ This model was merged using the [Model Stock](https://arxiv.org/abs/2403.19522) merge method using [Qwen/Qwen2.5-14B-Instruct](https://huggingface.co/Qwen/Qwen2.5-14B-Instruct) as a base.
133
+
134
+ ### Models Merged
135
+
136
+ The following models were included in the merge:
137
+ * [prithivMLmods/Calcium-Opus-14B-Elite](https://huggingface.co/prithivMLmods/Calcium-Opus-14B-Elite)
138
+ * [prithivMLmods/QwQ-LCoT-14B-Conversational](https://huggingface.co/prithivMLmods/QwQ-LCoT-14B-Conversational)
139
+
140
+ ### Configuration
141
+
142
+ The following YAML configuration was used to produce this model:
143
+
144
+ ```yaml
145
+ models:
146
+ - model: prithivMLmods/Calcium-Opus-14B-Elite
147
+ - model: prithivMLmods/QwQ-LCoT-14B-Conversational
148
+ merge_method: model_stock
149
+ base_model: Qwen/Qwen2.5-14B-Instruct
150
+ parameters:
151
+ normalize: false
152
+ int8_mask: true
153
+ dtype: bfloat16
154
+ tokenizer_source: "Qwen/Qwen2.5-14B-Instruct"
155
+ ```
156
+
157
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
158
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/prithivMLmods__Calcium-Opus-14B-Merge-details)!
159
+ Summarized results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/contents/viewer/default/train?q=prithivMLmods%2FCalcium-Opus-14B-Merge&sort[column]=Average%20%E2%AC%86%EF%B8%8F&sort[direction]=desc)!
160
+
161
+ | Metric |Value (%)|
162
+ |-------------------|--------:|
163
+ |**Average** | 35.80|
164
+ |IFEval (0-Shot) | 49.49|
165
+ |BBH (3-Shot) | 46.77|
166
+ |MATH Lvl 5 (4-Shot)| 33.08|
167
+ |GPQA (0-shot) | 16.11|
168
+ |MuSR (0-shot) | 20.93|
169
+ |MMLU-PRO (5-shot) | 48.40|
170
+