Sumail commited on
Commit
f1a4faf
·
verified ·
1 Parent(s): 284190d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +0 -50
README.md CHANGED
@@ -1,50 +0,0 @@
1
- ---
2
- base_model:
3
- - deepnetguy/gemma-118
4
- - tomaszki/gemma-42
5
- - rwh/gemma2
6
- library_name: transformers
7
- tags:
8
- - mergekit
9
- - merge
10
-
11
- ---
12
- # merge
13
-
14
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
15
-
16
- ## Merge Details
17
- ### Merge Method
18
-
19
- This model was merged using the [TIES](https://arxiv.org/abs/2306.01708) merge method using [tomaszki/gemma-42](https://huggingface.co/tomaszki/gemma-42) as a base.
20
-
21
- ### Models Merged
22
-
23
- The following models were included in the merge:
24
- * [deepnetguy/gemma-118](https://huggingface.co/deepnetguy/gemma-118)
25
- * [rwh/gemma2](https://huggingface.co/rwh/gemma2)
26
-
27
- ### Configuration
28
-
29
- The following YAML configuration was used to produce this model:
30
-
31
- ```yaml
32
-
33
- models:
34
- - model: tomaszki/gemma-42
35
- # no parameters necessary for base model
36
- - model: deepnetguy/gemma-118
37
- parameters:
38
- density: 0.5
39
- weight: 0.35
40
- - model: rwh/gemma2
41
- parameters:
42
- density: 0.5
43
- weight: 0.5
44
- merge_method: ties
45
- base_model: tomaszki/gemma-42
46
- parameters:
47
- normalize: true
48
- dtype: bfloat16
49
-
50
- ```