DELLA-Merging: Reducing Interference in Model Merging through Magnitude-Based Sampling
Paper • 2406.11617 • Published • 10
⚠️ Warning: This model can produce narratives and RP that contain violent and graphic erotic content. Adjust your system prompt accordingly, and use Mistral Tekken chat template.
Mistral Tekken chat template.
models:
- model: B:\24B\models--mistralai--Magistral-Small-2509\textonly
- model: B:\24B\models--Naphula--Slimaki-24B-v1
parameters:
weight: 0.4
density: 0.9
epsilon: 0.099
- model: B:\24B\models--DarkArtsForge--Magistaroth-24B-v1
parameters:
weight: 0.4
density: 0.9
epsilon: 0.099
- model: B:\24B\models--Casual-Autopsy--Maginum-Cydoms-24B
parameters:
weight: 0.4
density: 0.9
epsilon: 0.099
- model: B:\24B\models--sophosympatheia--Magistry-24B-v1.0
parameters:
weight: 0.4
density: 0.9
epsilon: 0.099
- model: B:\24B\models--TheDrummer--Precog-24B-v1
parameters:
weight: 0.4
density: 0.9
epsilon: 0.099
merge_method: della
base_model: B:\24B\models--mistralai--Magistral-Small-2509\textonly
parameters:
lambda: 1.0
normalize: false
tokenizer:
source: union
chat_template: auto
dtype: float32
out_dtype: bfloat16
name: 👺 Asmodeus-24B-v2