Ateron commited on
Commit
395f501
·
verified ·
1 Parent(s): ca1c60f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +44 -44
README.md CHANGED
@@ -1,44 +1,44 @@
1
- ---
2
- base_model: []
3
- library_name: transformers
4
- tags:
5
- - mergekit
6
- - merge
7
-
8
- ---
9
- # AI
10
-
11
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
12
-
13
- ## Merge Details
14
- ### Merge Method
15
-
16
- This model was merged using the [TIES](https://arxiv.org/abs/2306.01708) merge method using F:\AI\Pic\Magnum-Picaro-0.7-v2-12b as a base.
17
-
18
- ### Models Merged
19
-
20
- The following models were included in the merge:
21
- * F:\AI\Wayfarer\Wayfarer-12B
22
-
23
- ### Configuration
24
-
25
- The following YAML configuration was used to produce this model:
26
-
27
- ```yaml
28
- models:
29
- - model: F:\AI\Pic\Magnum-Picaro-0.7-v2-12b
30
- parameters:
31
- density: [1, 0.7, 0.1] # density gradient
32
- weight: 1.0
33
- - model: F:\AI\Wayfarer\Wayfarer-12B
34
- parameters:
35
- density: 0.3
36
- weight: [0, 0.3, 0.7, 1] # weight gradient
37
- merge_method: ties
38
- base_model: F:\AI\Pic\Magnum-Picaro-0.7-v2-12b
39
- parameters:
40
- normalize: true
41
- int8_mask: true
42
- dtype: float16
43
-
44
- ```
 
1
+ ---
2
+ base_model: []
3
+ library_name: transformers
4
+ tags:
5
+ - mergekit
6
+ - merge
7
+
8
+ ---
9
+ # AI
10
+
11
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
12
+
13
+ ## Merge Details
14
+ ### Merge Method
15
+
16
+ This model was merged using the [TIES](https://arxiv.org/abs/2306.01708) merge method using Magnum-Picaro-0.7-v2-12b as a base.
17
+
18
+ ### Models Merged
19
+
20
+ The following models were included in the merge:
21
+ * Wayfarer-12B
22
+
23
+ ### Configuration
24
+
25
+ The following YAML configuration was used to produce this model:
26
+
27
+ ```yaml
28
+ models:
29
+ - model: F:\AI\Pic\Magnum-Picaro-0.7-v2-12b
30
+ parameters:
31
+ density: [1, 0.7, 0.1] # density gradient
32
+ weight: 1.0
33
+ - model: F:\AI\Wayfarer\Wayfarer-12B
34
+ parameters:
35
+ density: 0.3
36
+ weight: [0, 0.3, 0.7, 1] # weight gradient
37
+ merge_method: ties
38
+ base_model: F:\AI\Pic\Magnum-Picaro-0.7-v2-12b
39
+ parameters:
40
+ normalize: true
41
+ int8_mask: true
42
+ dtype: float16
43
+
44
+ ```