Troy-Codes commited on
Commit
6f73b54
·
verified ·
1 Parent(s): ecca1a6

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -43
README.md CHANGED
@@ -1,43 +1 @@
1
- ---
2
- base_model: []
3
- library_name: transformers
4
- tags:
5
- - mergekit
6
- - merge
7
-
8
- ---
9
- # MoE-Evil
10
-
11
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
12
-
13
- ## Merge Details
14
- ### Merge Method
15
-
16
- This model was merged using the [Linear](https://arxiv.org/abs/2203.05482) merge method using examples/MoE-UAI as a base.
17
-
18
- ### Models Merged
19
-
20
- The following models were included in the merge:
21
- * examples/MoE-BASE_MOST_UNHINGED
22
-
23
- ### Configuration
24
-
25
- The following YAML configuration was used to produce this model:
26
-
27
- ```yaml
28
- models:
29
- - model: examples/MoE-BASE_MOST_UNHINGED
30
- parameters:
31
- weight: 1
32
- density: 1
33
- merge_method: linear
34
- base_model: examples/MoE-UAI
35
- parameters:
36
- weight: 1
37
- density: 1
38
- normalize: true
39
- int8_mask: true
40
- tokenizer_source: examples/MoE-UAI
41
- dtype: bfloat16
42
-
43
- ```
 
1
+ Have at it!