Sumail commited on
Commit
1228b6d
·
verified ·
1 Parent(s): 8f5d882

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +0 -50
README.md CHANGED
@@ -1,50 +0,0 @@
1
- ---
2
- base_model:
3
- - deepnetguy/gemma-109
4
- - deepnetguy/gemma-108
5
- - rwh/gemma2
6
- library_name: transformers
7
- tags:
8
- - mergekit
9
- - merge
10
-
11
- ---
12
- # merge
13
-
14
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
15
-
16
- ## Merge Details
17
- ### Merge Method
18
-
19
- This model was merged using the [TIES](https://arxiv.org/abs/2306.01708) merge method using [deepnetguy/gemma-108](https://huggingface.co/deepnetguy/gemma-108) as a base.
20
-
21
- ### Models Merged
22
-
23
- The following models were included in the merge:
24
- * [deepnetguy/gemma-109](https://huggingface.co/deepnetguy/gemma-109)
25
- * [rwh/gemma2](https://huggingface.co/rwh/gemma2)
26
-
27
- ### Configuration
28
-
29
- The following YAML configuration was used to produce this model:
30
-
31
- ```yaml
32
-
33
- models:
34
- - model: deepnetguy/gemma-108
35
- # no parameters necessary for base model
36
- - model: deepnetguy/gemma-109
37
- parameters:
38
- density: 0.5
39
- weight: 0.5
40
- - model: rwh/gemma2
41
- parameters:
42
- density: 0.5
43
- weight: 0.35
44
- merge_method: ties
45
- base_model: deepnetguy/gemma-108
46
- parameters:
47
- normalize: true
48
- dtype: bfloat16
49
-
50
- ```