Hugging Face's logo

mllm-dev
/
gpt2_m_experiment

Text Classification
Transformers
Safetensors
gpt2
mergekit
Merge
Model card Files
xet
Community
mllm-dev commited on
Commit
b006b06
·
verified ·
1 Parent(s): 68b4518

Upload folder using huggingface_hub

Browse files
Files changed (3) hide show
  1. README.md +19 -23
  2. mergekit_config.yml +9 -13
  3. model-00001-of-00001.safetensors +1 -1
README.md CHANGED
@@ -1,14 +1,14 @@
1
  ---
2
  base_model:
 
3
  - mllm-dev/gpt2_f_experiment_3
4
- - mllm-dev/gpt2_f_experiment_7
5
  - mllm-dev/gpt2_f_experiment_0
6
- - mllm-dev/gpt2_f_experiment_2
7
- - mllm-dev/gpt2_f_experiment_1
8
  - mllm-dev/gpt2_f_experiment_8
9
- - mllm-dev/gpt2_f_experiment_9
10
  - mllm-dev/gpt2_f_experiment_5
11
- - mllm-dev/gpt2_f_experiment_6
12
  - mllm-dev/gpt2_f_experiment_4
13
  library_name: transformers
14
  tags:
@@ -28,14 +28,14 @@ This model was merged using the [TIES](https://arxiv.org/abs/2306.01708) merge m
28
  ### Models Merged
29
 
30
  The following models were included in the merge:
31
- * [mllm-dev/gpt2_f_experiment_3](https://huggingface.co/mllm-dev/gpt2_f_experiment_3)
32
- * [mllm-dev/gpt2_f_experiment_7](https://huggingface.co/mllm-dev/gpt2_f_experiment_7)
33
  * [mllm-dev/gpt2_f_experiment_2](https://huggingface.co/mllm-dev/gpt2_f_experiment_2)
34
- * [mllm-dev/gpt2_f_experiment_1](https://huggingface.co/mllm-dev/gpt2_f_experiment_1)
35
- * [mllm-dev/gpt2_f_experiment_8](https://huggingface.co/mllm-dev/gpt2_f_experiment_8)
36
  * [mllm-dev/gpt2_f_experiment_9](https://huggingface.co/mllm-dev/gpt2_f_experiment_9)
37
- * [mllm-dev/gpt2_f_experiment_5](https://huggingface.co/mllm-dev/gpt2_f_experiment_5)
38
  * [mllm-dev/gpt2_f_experiment_6](https://huggingface.co/mllm-dev/gpt2_f_experiment_6)
 
 
 
 
39
  * [mllm-dev/gpt2_f_experiment_4](https://huggingface.co/mllm-dev/gpt2_f_experiment_4)
40
 
41
  ### Configuration
@@ -49,7 +49,6 @@ base_model:
49
  dtype: float16
50
  merge_method: ties
51
  parameters:
52
- int8_mask: 0.0
53
  normalize: 1.0
54
  slices:
55
  - sources:
@@ -57,70 +56,67 @@ slices:
57
  model:
58
  model:
59
  path: mllm-dev/gpt2_f_experiment_0
60
- parameters:
61
- density: 0.1
62
- weight: 0.1
63
  - layer_range: [0, 12]
64
  model:
65
  model:
66
  path: mllm-dev/gpt2_f_experiment_1
67
  parameters:
68
- density: 0.1
69
  weight: 0.1
70
  - layer_range: [0, 12]
71
  model:
72
  model:
73
  path: mllm-dev/gpt2_f_experiment_2
74
  parameters:
75
- density: 0.1
76
  weight: 0.1
77
  - layer_range: [0, 12]
78
  model:
79
  model:
80
  path: mllm-dev/gpt2_f_experiment_3
81
  parameters:
82
- density: 0.1
83
  weight: 0.1
84
  - layer_range: [0, 12]
85
  model:
86
  model:
87
  path: mllm-dev/gpt2_f_experiment_4
88
  parameters:
89
- density: 0.1
90
  weight: 0.1
91
  - layer_range: [0, 12]
92
  model:
93
  model:
94
  path: mllm-dev/gpt2_f_experiment_5
95
  parameters:
96
- density: 0.1
97
  weight: 0.1
98
  - layer_range: [0, 12]
99
  model:
100
  model:
101
  path: mllm-dev/gpt2_f_experiment_6
102
  parameters:
103
- density: 0.1
104
  weight: 0.1
105
  - layer_range: [0, 12]
106
  model:
107
  model:
108
  path: mllm-dev/gpt2_f_experiment_7
109
  parameters:
110
- density: 0.1
111
  weight: 0.1
112
  - layer_range: [0, 12]
113
  model:
114
  model:
115
  path: mllm-dev/gpt2_f_experiment_8
116
  parameters:
117
- density: 0.1
118
  weight: 0.1
119
  - layer_range: [0, 12]
120
  model:
121
  model:
122
  path: mllm-dev/gpt2_f_experiment_9
123
  parameters:
124
- density: 0.1
125
  weight: 0.1
126
  ```
 
1
  ---
2
  base_model:
3
+ - mllm-dev/gpt2_f_experiment_2
4
  - mllm-dev/gpt2_f_experiment_3
5
+ - mllm-dev/gpt2_f_experiment_9
6
  - mllm-dev/gpt2_f_experiment_0
7
+ - mllm-dev/gpt2_f_experiment_6
 
8
  - mllm-dev/gpt2_f_experiment_8
9
+ - mllm-dev/gpt2_f_experiment_7
10
  - mllm-dev/gpt2_f_experiment_5
11
+ - mllm-dev/gpt2_f_experiment_1
12
  - mllm-dev/gpt2_f_experiment_4
13
  library_name: transformers
14
  tags:
 
28
  ### Models Merged
29
 
30
  The following models were included in the merge:
 
 
31
  * [mllm-dev/gpt2_f_experiment_2](https://huggingface.co/mllm-dev/gpt2_f_experiment_2)
32
+ * [mllm-dev/gpt2_f_experiment_3](https://huggingface.co/mllm-dev/gpt2_f_experiment_3)
 
33
  * [mllm-dev/gpt2_f_experiment_9](https://huggingface.co/mllm-dev/gpt2_f_experiment_9)
 
34
  * [mllm-dev/gpt2_f_experiment_6](https://huggingface.co/mllm-dev/gpt2_f_experiment_6)
35
+ * [mllm-dev/gpt2_f_experiment_8](https://huggingface.co/mllm-dev/gpt2_f_experiment_8)
36
+ * [mllm-dev/gpt2_f_experiment_7](https://huggingface.co/mllm-dev/gpt2_f_experiment_7)
37
+ * [mllm-dev/gpt2_f_experiment_5](https://huggingface.co/mllm-dev/gpt2_f_experiment_5)
38
+ * [mllm-dev/gpt2_f_experiment_1](https://huggingface.co/mllm-dev/gpt2_f_experiment_1)
39
  * [mllm-dev/gpt2_f_experiment_4](https://huggingface.co/mllm-dev/gpt2_f_experiment_4)
40
 
41
  ### Configuration
 
49
  dtype: float16
50
  merge_method: ties
51
  parameters:
 
52
  normalize: 1.0
53
  slices:
54
  - sources:
 
56
  model:
57
  model:
58
  path: mllm-dev/gpt2_f_experiment_0
 
 
 
59
  - layer_range: [0, 12]
60
  model:
61
  model:
62
  path: mllm-dev/gpt2_f_experiment_1
63
  parameters:
64
+ density: 0.9
65
  weight: 0.1
66
  - layer_range: [0, 12]
67
  model:
68
  model:
69
  path: mllm-dev/gpt2_f_experiment_2
70
  parameters:
71
+ density: 0.9
72
  weight: 0.1
73
  - layer_range: [0, 12]
74
  model:
75
  model:
76
  path: mllm-dev/gpt2_f_experiment_3
77
  parameters:
78
+ density: 0.9
79
  weight: 0.1
80
  - layer_range: [0, 12]
81
  model:
82
  model:
83
  path: mllm-dev/gpt2_f_experiment_4
84
  parameters:
85
+ density: 0.9
86
  weight: 0.1
87
  - layer_range: [0, 12]
88
  model:
89
  model:
90
  path: mllm-dev/gpt2_f_experiment_5
91
  parameters:
92
+ density: 0.9
93
  weight: 0.1
94
  - layer_range: [0, 12]
95
  model:
96
  model:
97
  path: mllm-dev/gpt2_f_experiment_6
98
  parameters:
99
+ density: 0.9
100
  weight: 0.1
101
  - layer_range: [0, 12]
102
  model:
103
  model:
104
  path: mllm-dev/gpt2_f_experiment_7
105
  parameters:
106
+ density: 0.9
107
  weight: 0.1
108
  - layer_range: [0, 12]
109
  model:
110
  model:
111
  path: mllm-dev/gpt2_f_experiment_8
112
  parameters:
113
+ density: 0.9
114
  weight: 0.1
115
  - layer_range: [0, 12]
116
  model:
117
  model:
118
  path: mllm-dev/gpt2_f_experiment_9
119
  parameters:
120
+ density: 0.9
121
  weight: 0.1
122
  ```
mergekit_config.yml CHANGED
@@ -4,7 +4,6 @@ base_model:
4
  dtype: float16
5
  merge_method: ties
6
  parameters:
7
- int8_mask: 0.0
8
  normalize: 1.0
9
  slices:
10
  - sources:
@@ -12,69 +11,66 @@ slices:
12
  model:
13
  model:
14
  path: mllm-dev/gpt2_f_experiment_0
15
- parameters:
16
- density: 0.1
17
- weight: 0.1
18
  - layer_range: [0, 12]
19
  model:
20
  model:
21
  path: mllm-dev/gpt2_f_experiment_1
22
  parameters:
23
- density: 0.1
24
  weight: 0.1
25
  - layer_range: [0, 12]
26
  model:
27
  model:
28
  path: mllm-dev/gpt2_f_experiment_2
29
  parameters:
30
- density: 0.1
31
  weight: 0.1
32
  - layer_range: [0, 12]
33
  model:
34
  model:
35
  path: mllm-dev/gpt2_f_experiment_3
36
  parameters:
37
- density: 0.1
38
  weight: 0.1
39
  - layer_range: [0, 12]
40
  model:
41
  model:
42
  path: mllm-dev/gpt2_f_experiment_4
43
  parameters:
44
- density: 0.1
45
  weight: 0.1
46
  - layer_range: [0, 12]
47
  model:
48
  model:
49
  path: mllm-dev/gpt2_f_experiment_5
50
  parameters:
51
- density: 0.1
52
  weight: 0.1
53
  - layer_range: [0, 12]
54
  model:
55
  model:
56
  path: mllm-dev/gpt2_f_experiment_6
57
  parameters:
58
- density: 0.1
59
  weight: 0.1
60
  - layer_range: [0, 12]
61
  model:
62
  model:
63
  path: mllm-dev/gpt2_f_experiment_7
64
  parameters:
65
- density: 0.1
66
  weight: 0.1
67
  - layer_range: [0, 12]
68
  model:
69
  model:
70
  path: mllm-dev/gpt2_f_experiment_8
71
  parameters:
72
- density: 0.1
73
  weight: 0.1
74
  - layer_range: [0, 12]
75
  model:
76
  model:
77
  path: mllm-dev/gpt2_f_experiment_9
78
  parameters:
79
- density: 0.1
80
  weight: 0.1
 
4
  dtype: float16
5
  merge_method: ties
6
  parameters:
 
7
  normalize: 1.0
8
  slices:
9
  - sources:
 
11
  model:
12
  model:
13
  path: mllm-dev/gpt2_f_experiment_0
 
 
 
14
  - layer_range: [0, 12]
15
  model:
16
  model:
17
  path: mllm-dev/gpt2_f_experiment_1
18
  parameters:
19
+ density: 0.9
20
  weight: 0.1
21
  - layer_range: [0, 12]
22
  model:
23
  model:
24
  path: mllm-dev/gpt2_f_experiment_2
25
  parameters:
26
+ density: 0.9
27
  weight: 0.1
28
  - layer_range: [0, 12]
29
  model:
30
  model:
31
  path: mllm-dev/gpt2_f_experiment_3
32
  parameters:
33
+ density: 0.9
34
  weight: 0.1
35
  - layer_range: [0, 12]
36
  model:
37
  model:
38
  path: mllm-dev/gpt2_f_experiment_4
39
  parameters:
40
+ density: 0.9
41
  weight: 0.1
42
  - layer_range: [0, 12]
43
  model:
44
  model:
45
  path: mllm-dev/gpt2_f_experiment_5
46
  parameters:
47
+ density: 0.9
48
  weight: 0.1
49
  - layer_range: [0, 12]
50
  model:
51
  model:
52
  path: mllm-dev/gpt2_f_experiment_6
53
  parameters:
54
+ density: 0.9
55
  weight: 0.1
56
  - layer_range: [0, 12]
57
  model:
58
  model:
59
  path: mllm-dev/gpt2_f_experiment_7
60
  parameters:
61
+ density: 0.9
62
  weight: 0.1
63
  - layer_range: [0, 12]
64
  model:
65
  model:
66
  path: mllm-dev/gpt2_f_experiment_8
67
  parameters:
68
+ density: 0.9
69
  weight: 0.1
70
  - layer_range: [0, 12]
71
  model:
72
  model:
73
  path: mllm-dev/gpt2_f_experiment_9
74
  parameters:
75
+ density: 0.9
76
  weight: 0.1
model-00001-of-00001.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:2ec69e8ae66c7de30c52d601ba7fd250b91c6837903b62dea681f384548b6503
3
  size 248902264
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ac2d223e322da400f17b562a754f1a096a848b97d0f1bc42731b7a370dc0d4ef
3
  size 248902264