Hugging Face's logo

afterpartyjohn
/
sn11_submission3

Text Generation
Transformers
Safetensors
stablelm
mergekit
Merge
Model card Files
xet
 Community
afterpartyjohn commited on
Commit
e90ccdb
·
verified ·
1 Parent(s): 2110d15

Upload model

Browse files
Files changed (6) hide show
  1. README.md +34 -7
  2. config.json +1 -1
  3. mergekit_config.yml +24 -3
  4. model-00001-of-00001.safetensors +1 -1
  5. tokenizer.json +2 -2
  6. tokenizer_config.json +1 -0
README.md CHANGED
@@ -1,28 +1,34 @@
1
  ---
2
  base_model:
 
3
  - starnet/19star03
4
- - lionking927/s11-m1-0303
 
5
  - starnet/15star03
 
6
  library_name: transformers
7
  tags:
8
  - mergekit
9
  - merge
10
 
11
  ---
12
- # output
13
 
14
  This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
15
 
16
  ## Merge Details
17
  ### Merge Method
18
 
19
- This model was merged using the [TIES](https://arxiv.org/abs/2306.01708) merge method using [starnet/15star03](https://huggingface.co/starnet/15star03) as a base.
20
 
21
  ### Models Merged
22
 
23
  The following models were included in the merge:
 
24
  * [starnet/19star03](https://huggingface.co/starnet/19star03)
25
- * [lionking927/s11-m1-0303](https://huggingface.co/lionking927/s11-m1-0303)
 
 
26
 
27
  ### Configuration
28
 
@@ -30,7 +36,7 @@ The following YAML configuration was used to produce this model:
30
 
31
  ```yaml
32
  models:
33
- - model: starnet/15star03
34
  parameters:
35
  density: [1, 0.7, 0.1] # density gradient
36
  weight: 1.0
@@ -38,7 +44,28 @@ models:
38
  parameters:
39
  density: 0.5
40
  weight: [0, 0.3, 0.7, 1] # weight gradient
41
- - model: lionking927/s11-m1-0303
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
42
  parameters:
43
  density: 0.33
44
  weight:
@@ -46,7 +73,7 @@ models:
46
  value: 0.5
47
  - value: 0
48
  merge_method: ties
49
- base_model: starnet/15star03
50
  parameters:
51
  normalize: true
52
  int8_mask: true
 
1
  ---
2
  base_model:
3
+ - aks1s/13Aks-18
4
  - starnet/19star03
5
+ - OwOpeepeepoopoo/ZZZZZsubmission7
6
+ - OwOpeepeepoopoo/ZZZZZsubmission5
7
  - starnet/15star03
8
+ - irusl/05Ir-4
9
  library_name: transformers
10
  tags:
11
  - mergekit
12
  - merge
13
 
14
  ---
15
+ # output_2
16
 
17
  This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
18
 
19
  ## Merge Details
20
  ### Merge Method
21
 
22
+ This model was merged using the [TIES](https://arxiv.org/abs/2306.01708) merge method using [irusl/05Ir-4](https://huggingface.co/irusl/05Ir-4) as a base.
23
 
24
  ### Models Merged
25
 
26
  The following models were included in the merge:
27
+ * [aks1s/13Aks-18](https://huggingface.co/aks1s/13Aks-18)
28
  * [starnet/19star03](https://huggingface.co/starnet/19star03)
29
+ * [OwOpeepeepoopoo/ZZZZZsubmission7](https://huggingface.co/OwOpeepeepoopoo/ZZZZZsubmission7)
30
+ * [OwOpeepeepoopoo/ZZZZZsubmission5](https://huggingface.co/OwOpeepeepoopoo/ZZZZZsubmission5)
31
+ * [starnet/15star03](https://huggingface.co/starnet/15star03)
32
 
33
  ### Configuration
34
 
 
36
 
37
  ```yaml
38
  models:
39
+ - model: irusl/05Ir-4
40
  parameters:
41
  density: [1, 0.7, 0.1] # density gradient
42
  weight: 1.0
 
44
  parameters:
45
  density: 0.5
46
  weight: [0, 0.3, 0.7, 1] # weight gradient
47
+ - model: starnet/15star03
48
+ parameters:
49
+ density: 0.33
50
+ weight:
51
+ - filter: mlp
52
+ value: 0.5
53
+ - value: 0
54
+ - model: aks1s/13Aks-18
55
+ parameters:
56
+ density: 0.33
57
+ weight:
58
+ - filter: mlp
59
+ value: 0.5
60
+ - value: 0
61
+ - model: OwOpeepeepoopoo/ZZZZZsubmission7
62
+ parameters:
63
+ density: 0.33
64
+ weight:
65
+ - filter: mlp
66
+ value: 0.5
67
+ - value: 0
68
+ - model: OwOpeepeepoopoo/ZZZZZsubmission5
69
  parameters:
70
  density: 0.33
71
  weight:
 
73
  value: 0.5
74
  - value: 0
75
  merge_method: ties
76
+ base_model: irusl/05Ir-4
77
  parameters:
78
  normalize: true
79
  int8_mask: true
config.json CHANGED
@@ -1,5 +1,5 @@
1
  {
2
- "_name_or_path": "starnet/15star03",
3
  "architectures": [
4
  "StableLmForCausalLM"
5
  ],
 
1
  {
2
+ "_name_or_path": "irusl/05Ir-4",
3
  "architectures": [
4
  "StableLmForCausalLM"
5
  ],
mergekit_config.yml CHANGED
@@ -1,5 +1,5 @@
1
  models:
2
- - model: starnet/15star03
3
  parameters:
4
  density: [1, 0.7, 0.1] # density gradient
5
  weight: 1.0
@@ -7,7 +7,28 @@ models:
7
  parameters:
8
  density: 0.5
9
  weight: [0, 0.3, 0.7, 1] # weight gradient
10
- - model: lionking927/s11-m1-0303
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
11
  parameters:
12
  density: 0.33
13
  weight:
@@ -15,7 +36,7 @@ models:
15
  value: 0.5
16
  - value: 0
17
  merge_method: ties
18
- base_model: starnet/15star03
19
  parameters:
20
  normalize: true
21
  int8_mask: true
 
1
  models:
2
+ - model: irusl/05Ir-4
3
  parameters:
4
  density: [1, 0.7, 0.1] # density gradient
5
  weight: 1.0
 
7
  parameters:
8
  density: 0.5
9
  weight: [0, 0.3, 0.7, 1] # weight gradient
10
+ - model: starnet/15star03
11
+ parameters:
12
+ density: 0.33
13
+ weight:
14
+ - filter: mlp
15
+ value: 0.5
16
+ - value: 0
17
+ - model: aks1s/13Aks-18
18
+ parameters:
19
+ density: 0.33
20
+ weight:
21
+ - filter: mlp
22
+ value: 0.5
23
+ - value: 0
24
+ - model: OwOpeepeepoopoo/ZZZZZsubmission7
25
+ parameters:
26
+ density: 0.33
27
+ weight:
28
+ - filter: mlp
29
+ value: 0.5
30
+ - value: 0
31
+ - model: OwOpeepeepoopoo/ZZZZZsubmission5
32
  parameters:
33
  density: 0.33
34
  weight:
 
36
  value: 0.5
37
  - value: 0
38
  merge_method: ties
39
+ base_model: irusl/05Ir-4
40
  parameters:
41
  normalize: true
42
  int8_mask: true
model-00001-of-00001.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:0a79c3d9cc401e05fc6565283c5dde1951cee752180b701a7d82c46f87a098df
3
  size 3289069184
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e739e741d073e503685db1a5c3be3acc11b18ef5306efa408d781b6d0579d2d4
3
  size 3289069184
tokenizer.json CHANGED
@@ -2,13 +2,13 @@
2
  "version": "1.0",
3
  "truncation": {
4
  "direction": "Right",
5
- "max_length": 3896,
6
  "strategy": "LongestFirst",
7
  "stride": 0
8
  },
9
  "padding": {
10
  "strategy": {
11
- "Fixed": 3896
12
  },
13
  "direction": "Left",
14
  "pad_to_multiple_of": null,
 
2
  "version": "1.0",
3
  "truncation": {
4
  "direction": "Right",
5
+ "max_length": 3696,
6
  "strategy": "LongestFirst",
7
  "stride": 0
8
  },
9
  "padding": {
10
  "strategy": {
11
+ "Fixed": 3696
12
  },
13
  "direction": "Left",
14
  "pad_to_multiple_of": null,
tokenizer_config.json CHANGED
@@ -1,4 +1,5 @@
1
  {
 
2
  "add_prefix_space": false,
3
  "added_tokens_decoder": {
4
  "100256": {
 
1
  {
2
+ "add_bos_token": true,
3
  "add_prefix_space": false,
4
  "added_tokens_decoder": {
5
  "100256": {