komleva commited on
Commit
efe4cf8
·
verified ·
1 Parent(s): 530a931

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +0 -45
README.md CHANGED
@@ -1,45 +0,0 @@
1
- ---
2
- base_model:
3
- - fatafix0000/aaa0
4
- - fatafix0000/aaa2
5
- library_name: transformers
6
- tags:
7
- - mergekit
8
- - merge
9
-
10
- ---
11
- # merged
12
-
13
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
14
-
15
- ## Merge Details
16
- ### Merge Method
17
-
18
- This model was merged using the [TIES](https://arxiv.org/abs/2306.01708) merge method using [fatafix0000/aaa0](https://huggingface.co/fatafix0000/aaa0) as a base.
19
-
20
- ### Models Merged
21
-
22
- The following models were included in the merge:
23
- * [fatafix0000/aaa2](https://huggingface.co/fatafix0000/aaa2)
24
-
25
- ### Configuration
26
-
27
- The following YAML configuration was used to produce this model:
28
-
29
- ```yaml
30
- base_model: fatafix0000/aaa0
31
- dtype: float16
32
- merge_method: ties
33
- slices:
34
- - sources:
35
- - layer_range: [0, 40]
36
- model: fatafix0000/aaa0
37
- parameters:
38
- density: [1.0, 0.7, 0.1]
39
- weight: 1.0
40
- - layer_range: [0, 40]
41
- model: fatafix0000/aaa2
42
- parameters:
43
- density: 0.33
44
- weight: 0.0001
45
- ```