| | --- |
| | base_model: |
| | - Nisk36/finetuned-lmsys_vicuna-7b-v1.5 |
| | - Nisk36/FT_elyza_ELYZA-japanese-Llama-2-7b-instruct |
| | library_name: transformers |
| | tags: |
| | - mergekit |
| | - merge |
| |
|
| | --- |
| | # final_model |
| | |
| | This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit). |
| | |
| | ## Merge Details |
| | ### Merge Method |
| | |
| | This model was merged using the [linear](https://arxiv.org/abs/2203.05482) merge method. |
| | |
| | ### Models Merged |
| | |
| | The following models were included in the merge: |
| | * [Nisk36/finetuned-lmsys_vicuna-7b-v1.5](https://huggingface.co/Nisk36/finetuned-lmsys_vicuna-7b-v1.5) |
| | * [Nisk36/FT_elyza_ELYZA-japanese-Llama-2-7b-instruct](https://huggingface.co/Nisk36/FT_elyza_ELYZA-japanese-Llama-2-7b-instruct) |
| | |
| | ### Configuration |
| | |
| | The following YAML configuration was used to produce this model: |
| | |
| | ```yaml |
| | dtype: bfloat16 |
| | merge_method: linear |
| | parameters: |
| | int8_mask: 1.0 |
| | normalize: 1.0 |
| | slices: |
| | - sources: |
| | - layer_range: [0, 4] |
| | model: Nisk36/finetuned-lmsys_vicuna-7b-v1.5 |
| | parameters: |
| | weight: 0.6235769265047518 |
| | - layer_range: [0, 4] |
| | model: Nisk36/FT_elyza_ELYZA-japanese-Llama-2-7b-instruct |
| | parameters: |
| | weight: 0.7274442555681364 |
| | - sources: |
| | - layer_range: [4, 8] |
| | model: Nisk36/finetuned-lmsys_vicuna-7b-v1.5 |
| | parameters: |
| | weight: 0.5271398694239577 |
| | - layer_range: [4, 8] |
| | model: Nisk36/FT_elyza_ELYZA-japanese-Llama-2-7b-instruct |
| | parameters: |
| | weight: 0.3489250438855029 |
| | - sources: |
| | - layer_range: [8, 12] |
| | model: Nisk36/finetuned-lmsys_vicuna-7b-v1.5 |
| | parameters: |
| | weight: 0.15496421762028023 |
| | - layer_range: [8, 12] |
| | model: Nisk36/FT_elyza_ELYZA-japanese-Llama-2-7b-instruct |
| | parameters: |
| | weight: 0.541330668871115 |
| | - sources: |
| | - layer_range: [12, 16] |
| | model: Nisk36/finetuned-lmsys_vicuna-7b-v1.5 |
| | parameters: |
| | weight: 0.5267269624685371 |
| | - layer_range: [12, 16] |
| | model: Nisk36/FT_elyza_ELYZA-japanese-Llama-2-7b-instruct |
| | parameters: |
| | weight: 0.8265113027826562 |
| | - sources: |
| | - layer_range: [16, 20] |
| | model: Nisk36/finetuned-lmsys_vicuna-7b-v1.5 |
| | parameters: |
| | weight: 0.6599861585345389 |
| | - layer_range: [16, 20] |
| | model: Nisk36/FT_elyza_ELYZA-japanese-Llama-2-7b-instruct |
| | parameters: |
| | weight: -0.249060520039947 |
| | - sources: |
| | - layer_range: [20, 24] |
| | model: Nisk36/finetuned-lmsys_vicuna-7b-v1.5 |
| | parameters: |
| | weight: 0.7761318532349375 |
| | - layer_range: [20, 24] |
| | model: Nisk36/FT_elyza_ELYZA-japanese-Llama-2-7b-instruct |
| | parameters: |
| | weight: 0.7040995904551324 |
| | - sources: |
| | - layer_range: [24, 28] |
| | model: Nisk36/finetuned-lmsys_vicuna-7b-v1.5 |
| | parameters: |
| | weight: 0.40152017541360374 |
| | - layer_range: [24, 28] |
| | model: Nisk36/FT_elyza_ELYZA-japanese-Llama-2-7b-instruct |
| | parameters: |
| | weight: 0.767141768059921 |
| | - sources: |
| | - layer_range: [28, 32] |
| | model: Nisk36/finetuned-lmsys_vicuna-7b-v1.5 |
| | parameters: |
| | weight: -0.004536646708608122 |
| | - layer_range: [28, 32] |
| | model: Nisk36/FT_elyza_ELYZA-japanese-Llama-2-7b-instruct |
| | parameters: |
| | weight: 0.8295357241419378 |
| | ``` |
| | |
