Naphula commited on
Commit
a127995
·
verified ·
1 Parent(s): 6c34e62

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +211 -1
README.md CHANGED
@@ -16,4 +16,214 @@ This was broken due to using `ties` and also censored
16
 
17
  # T11
18
  Testing this now with `della`
19
- ![T11](https://cdn-uploads.huggingface.co/production/uploads/68e840caa318194c44ec2a04/dD67BbjC6LfeVLXrJ5ehn.png)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
16
 
17
  # T11
18
  Testing this now with `della`
19
+ ![T11](https://cdn-uploads.huggingface.co/production/uploads/68e840caa318194c44ec2a04/dD67BbjC6LfeVLXrJ5ehn.png)
20
+
21
+ # Update 3
22
+ Here are the full yaml configs used to create this checkpoint
23
+
24
+ Asmodeus v2e
25
+ ```yaml
26
+ models:
27
+ - model: B:\24B\models--mistralai--Magistral-Small-2509\textonly
28
+ - model: B:\24B\models--Naphula--Slimaki-24B-v1
29
+ parameters:
30
+ weight: 0.3
31
+ density: 0.9
32
+ epsilon: 0.099
33
+ - model: B:\24B\models--DarkArtsForge--Magistaroth-24B-v1
34
+ parameters:
35
+ weight: 0.3
36
+ density: 0.9
37
+ epsilon: 0.099
38
+ - model: B:\24B\models--Casual-Autopsy--Maginum-Cydoms-24B
39
+ parameters:
40
+ weight: 0.3
41
+ density: 0.9
42
+ epsilon: 0.099
43
+ - model: B:\24B\models--sophosympatheia--Magistry-24B-v1.0
44
+ parameters:
45
+ weight: 0.3
46
+ density: 0.9
47
+ epsilon: 0.099
48
+ - model: B:\24B\!models--TheDrummer--Precog-24B-v1
49
+ parameters:
50
+ weight: 0.6
51
+ density: 0.9
52
+ epsilon: 0.099
53
+ - model: B:\24B\models--DarkArtsForge--Asmodeus-24B-v1
54
+ parameters:
55
+ weight: 0.3
56
+ density: 0.9
57
+ epsilon: 0.099
58
+ merge_method: della
59
+ base_model: B:\24B\models--mistralai--Magistral-Small-2509\textonly
60
+ parameters:
61
+ lambda: 1.0
62
+ normalize: false
63
+ tokenizer:
64
+ source: union
65
+ chat_template: auto
66
+ dtype: float32
67
+ out_dtype: bfloat16
68
+ ```
69
+
70
+ T5
71
+ ```yaml
72
+ architecture: MistralForCausalLM
73
+ models:
74
+ - model: B:\24B\models--anthracite-core--Mistral-Small-3.2-24B-Instruct-2506-Text-Only # B:\24B\models--mistralai--Magistral-Small-2509\textonly
75
+ - model: B:\24B\BeaverAI_Fallen-Mistral-Small-3.1-24B-v1e_textonly
76
+ parameters:
77
+ weight: 0.25
78
+ density: 0.9
79
+ epsilon: 0.099
80
+ - model: B:\24B\models--TheDrummer--Cydonia-24B-v4.3
81
+ parameters:
82
+ weight: 0.25
83
+ density: 0.9
84
+ epsilon: 0.099
85
+ - model: B:\24B\models--TheDrummer--Magidonia-24B-v4.3
86
+ parameters:
87
+ weight: 0.25
88
+ density: 0.9
89
+ epsilon: 0.099
90
+ - model: B:\24B\models--TheDrummer--Precog-24B-v1
91
+ parameters:
92
+ weight: 0.25
93
+ density: 0.9
94
+ epsilon: 0.099
95
+ - model: B:\24B\models--zerofata--MS3.2-PaintedFantasy-v2-24B
96
+ parameters:
97
+ weight: 0.25
98
+ density: 0.9
99
+ epsilon: 0.099
100
+ - model: B:\24B\models--zerofata--MS3.2-PaintedFantasy-v3-24B
101
+ parameters:
102
+ weight: 0.25
103
+ density: 0.9
104
+ epsilon: 0.099
105
+ - model: B:\24B\models--zerofata--MS3.2-PaintedFantasy-v4.1-24B
106
+ parameters:
107
+ weight: 0.25
108
+ density: 0.9
109
+ epsilon: 0.099
110
+ merge_method: della
111
+ base_model: B:\24B\models--anthracite-core--Mistral-Small-3.2-24B-Instruct-2506-Text-Only # B:\24B\models--mistralai--Magistral-Small-2509\textonly
112
+ parameters:
113
+ lambda: 1.0
114
+ normalize: false
115
+ int8_mask: false
116
+ dtype: float32
117
+ out_dtype: bfloat16
118
+ tokenizer:
119
+ source: union
120
+ chat_template: auto
121
+ ```
122
+
123
+ T6
124
+ ```yaml
125
+ models:
126
+ - model: B:\24B\models--mistralai--Magistral-Small-2509\textonly
127
+ - model: B:\24B\models--Naphula--Slimaki-24B-v1
128
+ parameters:
129
+ weight: 0.25
130
+ density: 0.9
131
+ epsilon: 0.099
132
+ - model: B:\24B\models--Casual-Autopsy--Maginum-Cydoms-24B
133
+ parameters:
134
+ weight: 0.25
135
+ density: 0.9
136
+ epsilon: 0.099
137
+ - model: B:\24B\models--Naphula--GhostFace-24B-v1
138
+ parameters:
139
+ weight: 0.25
140
+ density: 0.9
141
+ epsilon: 0.099
142
+ - model: B:\24B\DarkArtsForge__Asmodeus-24B-v2
143
+ parameters:
144
+ weight: 0.25
145
+ density: 0.9
146
+ epsilon: 0.099
147
+ - model: B:\24B\models--sophosympatheia--Magistry-24B-v1.0
148
+ parameters:
149
+ weight: 0.25
150
+ density: 0.9
151
+ epsilon: 0.099
152
+ - model: B:\24B\models--DarkArtsForge--Asmodeus-24B-v1
153
+ parameters:
154
+ weight: 0.25
155
+ density: 0.9
156
+ epsilon: 0.099
157
+ - model: C:\mergekit-main\T5
158
+ parameters:
159
+ weight: 0.25
160
+ density: 0.9
161
+ epsilon: 0.099
162
+ - model: B:\24B\Asmo_v2e
163
+ parameters:
164
+ weight: 0.25
165
+ density: 0.9
166
+ epsilon: 0.099
167
+ merge_method: della
168
+ base_model: B:\24B\models--mistralai--Magistral-Small-2509\textonly
169
+ parameters:
170
+ lambda: 1.0
171
+ normalize: false
172
+ tokenizer:
173
+ source: union
174
+ chat_template: auto
175
+ dtype: float32
176
+ out_dtype: bfloat16
177
+ ```
178
+
179
+ T7
180
+ ```yaml
181
+ models:
182
+ - model: B:\24B\models--mistralai--Magistral-Small-2509\textonly
183
+ - model: B:\24B\models--Naphula--Slimaki-24B-v1
184
+ parameters:
185
+ weight: 0.285
186
+ density: 0.9
187
+ epsilon: 0.099
188
+ - model: B:\24B\models--Casual-Autopsy--Maginum-Cydoms-24B
189
+ parameters:
190
+ weight: 0.285
191
+ density: 0.9
192
+ epsilon: 0.099
193
+ - model: B:\24B\DarkArtsForge__Asmodeus-24B-v2
194
+ parameters:
195
+ weight: 0.285
196
+ density: 0.9
197
+ epsilon: 0.099
198
+ - model: B:\24B\models--sophosympatheia--Magistry-24B-v1.0
199
+ parameters:
200
+ weight: 0.285
201
+ density: 0.9
202
+ epsilon: 0.099
203
+ - model: B:\24B\models--DarkArtsForge--Asmodeus-24B-v1
204
+ parameters:
205
+ weight: 0.285
206
+ density: 0.9
207
+ epsilon: 0.099
208
+ - model: B:\24B\Asmo_v2e
209
+ parameters:
210
+ weight: 0.285
211
+ density: 0.9
212
+ epsilon: 0.099
213
+ - model: B:\24B\T6
214
+ parameters:
215
+ weight: 0.285
216
+ density: 0.9
217
+ epsilon: 0.099
218
+ merge_method: della
219
+ base_model: B:\24B\models--mistralai--Magistral-Small-2509\textonly
220
+ parameters:
221
+ lambda: 1.0
222
+ normalize: false
223
+ tokenizer:
224
+ source: union
225
+ chat_template: auto
226
+ dtype: float32
227
+ out_dtype: bfloat16
228
+ name: Checkpoint-T7-24B
229
+ ```