schonsense commited on
Commit
14caf9a
·
verified ·
1 Parent(s): 58c87ef

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +58 -58
README.md CHANGED
@@ -1,58 +1,58 @@
1
- ---
2
- base_model:
3
- - ReadyArt/L3.3-The-Omega-Directive-70B-Unslop-v2.1
4
- - allura-org/Bigger-Body-70b
5
- - Mawdistical/RAWMAW-70B
6
- library_name: transformers
7
- tags:
8
- - mergekit
9
- - merge
10
-
11
- ---
12
- # sce_smoot_v2
13
-
14
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
15
-
16
- ## Merge Details
17
- ### Merge Method
18
-
19
- This model was merged using the [SCE](https://arxiv.org/abs/2408.07990) merge method using D:\mergekit\yamls\IPOplectic as a base.
20
-
21
- ### Models Merged
22
-
23
- The following models were included in the merge:
24
- * [ReadyArt/L3.3-The-Omega-Directive-70B-Unslop-v2.1](https://huggingface.co/ReadyArt/L3.3-The-Omega-Directive-70B-Unslop-v2.1)
25
- * [allura-org/Bigger-Body-70b](https://huggingface.co/allura-org/Bigger-Body-70b)
26
- * [Mawdistical/RAWMAW-70B](https://huggingface.co/Mawdistical/RAWMAW-70B)
27
-
28
- ### Configuration
29
-
30
- The following YAML configuration was used to produce this model:
31
-
32
- ```yaml
33
- merge_method: sce
34
- select_topk: 0.25
35
-
36
- models:
37
-
38
-
39
- - model: "D:\\mergekit\\yamls\\IPOplectic"
40
- - model: allura-org/Bigger-Body-70b
41
- - model: ReadyArt/L3.3-The-Omega-Directive-70B-Unslop-v2.1
42
- - model: Mawdistical/RAWMAW-70B
43
-
44
-
45
- base_model: "D:\\mergekit\\yamls\\IPOplectic"
46
-
47
- parameters:
48
- normalize: false
49
- int8_mask: true
50
-
51
- dtype: float32
52
- out_dtype: bfloat16
53
-
54
- tokenizer:
55
- source: schonsense/Tropoplectic
56
- pad_to_multiple_of: 8
57
-
58
- ```
 
1
+ ---
2
+ base_model:
3
+ - ReadyArt/L3.3-The-Omega-Directive-70B-Unslop-v2.1
4
+ - allura-org/Bigger-Body-70b
5
+ - Mawdistical/RAWMAW-70B
6
+ - schonsense/IPOplectic
7
+ library_name: transformers
8
+ tags:
9
+ - mergekit
10
+ - merge
11
+ ---
12
+ # sce_smoot_v2
13
+
14
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
15
+
16
+ ## Merge Details
17
+ ### Merge Method
18
+
19
+ This model was merged using the [SCE](https://arxiv.org/abs/2408.07990) merge method using D:\mergekit\yamls\IPOplectic as a base.
20
+
21
+ ### Models Merged
22
+
23
+ The following models were included in the merge:
24
+ * [ReadyArt/L3.3-The-Omega-Directive-70B-Unslop-v2.1](https://huggingface.co/ReadyArt/L3.3-The-Omega-Directive-70B-Unslop-v2.1)
25
+ * [allura-org/Bigger-Body-70b](https://huggingface.co/allura-org/Bigger-Body-70b)
26
+ * [Mawdistical/RAWMAW-70B](https://huggingface.co/Mawdistical/RAWMAW-70B)
27
+
28
+ ### Configuration
29
+
30
+ The following YAML configuration was used to produce this model:
31
+
32
+ ```yaml
33
+ merge_method: sce
34
+ select_topk: 0.25
35
+
36
+ models:
37
+
38
+
39
+ - model: "D:\\mergekit\\yamls\\IPOplectic"
40
+ - model: allura-org/Bigger-Body-70b
41
+ - model: ReadyArt/L3.3-The-Omega-Directive-70B-Unslop-v2.1
42
+ - model: Mawdistical/RAWMAW-70B
43
+
44
+
45
+ base_model: "D:\\mergekit\\yamls\\IPOplectic"
46
+
47
+ parameters:
48
+ normalize: false
49
+ int8_mask: true
50
+
51
+ dtype: float32
52
+ out_dtype: bfloat16
53
+
54
+ tokenizer:
55
+ source: schonsense/Tropoplectic
56
+ pad_to_multiple_of: 8
57
+
58
+ ```