Resolving Interference When Merging Models
Paper
• 2306.01708 • Published
• 17
This is quantized version of DreadPoor/Aspire1.2-8B-TIES created using llama.cpp
This is a merge of pre-trained language models created using mergekit.
This model was merged using the TIES merge method using NousResearch/Meta-Llama-3-8B as a base.
The following models were included in the merge:
The following YAML configuration was used to produce this model:
models:
- model: Orenguteng/Llama-3.1-8B-Lexi-Uncensored-V2+kloodia/lora-8b-bio
parameters:
weight: 1
- model: arcee-ai/Llama-3.1-SuperNova-Lite+Blackroot/Llama3-RP-Lora
parameters:
weight: 1
- model: NousResearch/Hermes-3-Llama-3.1-8B+kloodia/lora-8b-physic
parameters:
weight: 1
- model: cgato/L3-TheSpice-8b-v0.8.3+kloodia/lora-8b-medic
parameters:
weight: 1
- model: ArliAI/Llama-3.1-8B-ArliAI-RPMax-v1.1+Blackroot/Llama-3-8B-Abomination-LORA
parameters:
weight: 1
- model: DreadPoor/Nothing_to_see_here_-_Move_along+hikikomoriHaven/llama3-8b-hikikomori-v0.4
parameters:
weight: 1
merge_method: ties
base_model: NousResearch/Meta-Llama-3-8B
parameters:
density: 1
normalize: true
int8_mask: true
dtype: bfloat16
2-bit
3-bit
4-bit
5-bit
6-bit
8-bit