Sumail commited on
Commit
cdcda84
·
verified ·
1 Parent(s): 86da313

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +0 -57
README.md CHANGED
@@ -1,57 +0,0 @@
1
- ---
2
- base_model:
3
- - tomaszki/gemma-39
4
- - rwh/gemma2
5
- - deepnetguy/gemma-108
6
- - heyllm234/sn6_models
7
- library_name: transformers
8
- tags:
9
- - mergekit
10
- - merge
11
-
12
- ---
13
- # merge
14
-
15
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
16
-
17
- ## Merge Details
18
- ### Merge Method
19
-
20
- This model was merged using the [DARE](https://arxiv.org/abs/2311.03099) [TIES](https://arxiv.org/abs/2306.01708) merge method using [tomaszki/gemma-39](https://huggingface.co/tomaszki/gemma-39) as a base.
21
-
22
- ### Models Merged
23
-
24
- The following models were included in the merge:
25
- * [rwh/gemma2](https://huggingface.co/rwh/gemma2)
26
- * [deepnetguy/gemma-108](https://huggingface.co/deepnetguy/gemma-108)
27
- * [heyllm234/sn6_models](https://huggingface.co/heyllm234/sn6_models)
28
-
29
- ### Configuration
30
-
31
- The following YAML configuration was used to produce this model:
32
-
33
- ```yaml
34
-
35
- models:
36
- - model: tomaszki/gemma-39
37
- # No parameters necessary for base model
38
- - model: deepnetguy/gemma-108
39
- parameters:
40
- density: 0.53
41
- weight: 0.3
42
- - model: rwh/gemma2
43
- parameters:
44
- density: 0.53
45
- weight: 0.4
46
- - model: heyllm234/sn6_models
47
- parameters:
48
- density: 0.53
49
- weight: 0.35
50
- merge_method: dare_ties
51
- base_model: tomaszki/gemma-39
52
- parameters:
53
- int8_mask: true
54
- dtype: bfloat16
55
-
56
-
57
- ```