Delta-Vector commited on
Commit
b1d3415
·
verified ·
1 Parent(s): cdc7474

Upload ./README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +54 -0
README.md ADDED
@@ -0,0 +1,54 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model:
3
+ - TheDrummer/UnslopSmall-22B-v1
4
+ - NewEden-Staging/Kyne-22b
5
+ - nbeerbower/Mistral-Small-Gutenberg-Doppel-22B
6
+ library_name: transformers
7
+ tags:
8
+ - mergekit
9
+ - merge
10
+
11
+ ---
12
+ ### exl2 quant (measurement.json in main branch)
13
+ ---
14
+ ### check revisions for quants
15
+ ---
16
+
17
+ # merge
18
+
19
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
20
+
21
+ ## Merge Details
22
+ ### Merge Method
23
+
24
+ This model was merged using the [task arithmetic](https://arxiv.org/abs/2212.04089) merge method using [NewEden-Staging/Kyne-22b](https://huggingface.co/NewEden-Staging/Kyne-22b) as a base.
25
+
26
+ ### Models Merged
27
+
28
+ The following models were included in the merge:
29
+ * [TheDrummer/UnslopSmall-22B-v1](https://huggingface.co/TheDrummer/UnslopSmall-22B-v1)
30
+ * [nbeerbower/Mistral-Small-Gutenberg-Doppel-22B](https://huggingface.co/nbeerbower/Mistral-Small-Gutenberg-Doppel-22B)
31
+
32
+ ### Configuration
33
+
34
+ The following YAML configuration was used to produce this model:
35
+
36
+ ```yaml
37
+ models:
38
+ - model: TheDrummer/UnslopSmall-22B-v1
39
+ parameters:
40
+ density: 0.7
41
+ weight: 0.7
42
+ - model: nbeerbower/Mistral-Small-Gutenberg-Doppel-22B
43
+ parameters:
44
+ density: 0.3
45
+ weight: 0.3
46
+
47
+ merge_method: task_arithmetic
48
+ base_model: NewEden-Staging/Kyne-22b
49
+ parameters:
50
+ normalize: false
51
+ int8_mask: true
52
+ dtype: bfloat16
53
+
54
+ ```