File size: 3,263 Bytes
3d37ab0 fd99836 d3b567f c0c0028 3813008 d3b567f 3813008 d3b567f 3813008 d3b567f 3d37ab0 3813008 3d37ab0 fd99836 d3b567f 3813008 d3b567f 3813008 d3b567f 3813008 d3b567f 3d37ab0 c0c0028 3d37ab0 3813008 c0c0028 3d37ab0 3813008 c0c0028 3d37ab0 3813008 c0c0028 3d37ab0 3813008 c0c0028 3d37ab0 3813008 c0c0028 8629720 3813008 c0c0028 8629720 3813008 c0c0028 8629720 3813008 c0c0028 8629720 3813008 c0c0028 8629720 3813008 c0c0028 3d37ab0 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 |
---
base_model:
- mllm-dev/gpt2_f_experiment_7
- mllm-dev/gpt2_f_experiment_1
- mllm-dev/gpt2_f_experiment_0
- mllm-dev/gpt2_f_experiment_8
- mllm-dev/gpt2_f_experiment_2
- mllm-dev/gpt2_f_experiment_4
- mllm-dev/gpt2_f_experiment_5
- mllm-dev/gpt2_f_experiment_6
- mllm-dev/gpt2_f_experiment_3
- mllm-dev/gpt2_f_experiment_9
library_name: transformers
tags:
- mergekit
- merge
---
# sean_test_merge_out
This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
## Merge Details
### Merge Method
This model was merged using the [DARE](https://arxiv.org/abs/2311.03099) [TIES](https://arxiv.org/abs/2306.01708) merge method using [mllm-dev/gpt2_f_experiment_0](https://huggingface.co/mllm-dev/gpt2_f_experiment_0) as a base.
### Models Merged
The following models were included in the merge:
* [mllm-dev/gpt2_f_experiment_7](https://huggingface.co/mllm-dev/gpt2_f_experiment_7)
* [mllm-dev/gpt2_f_experiment_1](https://huggingface.co/mllm-dev/gpt2_f_experiment_1)
* [mllm-dev/gpt2_f_experiment_8](https://huggingface.co/mllm-dev/gpt2_f_experiment_8)
* [mllm-dev/gpt2_f_experiment_2](https://huggingface.co/mllm-dev/gpt2_f_experiment_2)
* [mllm-dev/gpt2_f_experiment_4](https://huggingface.co/mllm-dev/gpt2_f_experiment_4)
* [mllm-dev/gpt2_f_experiment_5](https://huggingface.co/mllm-dev/gpt2_f_experiment_5)
* [mllm-dev/gpt2_f_experiment_6](https://huggingface.co/mllm-dev/gpt2_f_experiment_6)
* [mllm-dev/gpt2_f_experiment_3](https://huggingface.co/mllm-dev/gpt2_f_experiment_3)
* [mllm-dev/gpt2_f_experiment_9](https://huggingface.co/mllm-dev/gpt2_f_experiment_9)
### Configuration
The following YAML configuration was used to produce this model:
```yaml
base_model:
model:
path: mllm-dev/gpt2_f_experiment_0
dtype: float16
merge_method: dare_ties
parameters:
normalize: 1.0
slices:
- sources:
- layer_range: [0, 12]
model:
model:
path: mllm-dev/gpt2_f_experiment_0
- layer_range: [0, 12]
model:
model:
path: mllm-dev/gpt2_f_experiment_1
parameters:
density: 0.9
weight: 0.1
- layer_range: [0, 12]
model:
model:
path: mllm-dev/gpt2_f_experiment_2
parameters:
density: 0.9
weight: 0.1
- layer_range: [0, 12]
model:
model:
path: mllm-dev/gpt2_f_experiment_3
parameters:
density: 0.9
weight: 0.1
- layer_range: [0, 12]
model:
model:
path: mllm-dev/gpt2_f_experiment_4
parameters:
density: 0.9
weight: 0.1
- layer_range: [0, 12]
model:
model:
path: mllm-dev/gpt2_f_experiment_5
parameters:
density: 0.9
weight: 0.1
- layer_range: [0, 12]
model:
model:
path: mllm-dev/gpt2_f_experiment_6
parameters:
density: 0.9
weight: 0.1
- layer_range: [0, 12]
model:
model:
path: mllm-dev/gpt2_f_experiment_7
parameters:
density: 0.9
weight: 0.1
- layer_range: [0, 12]
model:
model:
path: mllm-dev/gpt2_f_experiment_8
parameters:
density: 0.9
weight: 0.1
- layer_range: [0, 12]
model:
model:
path: mllm-dev/gpt2_f_experiment_9
parameters:
density: 0.9
weight: 0.1
```
|