metadata
base_model:
- Qwen/Qwen2.5-1.5B
- Qwen/Qwen2.5-1.5B-Instruct
library_name: transformers
tags:
- mergekit
- merge
- ties
- qwen
- qwen-2.5
license: apache-2.0
model_type: causal-lm
pipeline_tag: text-generation
Qwen2.5-1.5B-TIES-Merge
This repository contains a 1.5 billion parameter model created by merging Qwen2.5-1.5B (Base) and Qwen2.5-1.5B-Instruct using the TIES (Trim, Elect Sign & Merge) method.
Model Highlights
- Logic Stability: Uses TIES merging to resolve parameter conflicts, resulting in better arithmetic and reasoning than standard SLERP merges.
- Architecture: 24 layers, 1.5B parameters.
The following models were included in the merge:
Configuration
The merge was performed using Mergekit. The following YAML configuration was used to produce this model:
models:
- model: Qwen/Qwen2.5-1.5B
parameters:
density: 0.5
weight: 0.5
- model: Qwen/Qwen2.5-1.5B-Instruct
parameters:
density: 0.5
weight: 0.5
merge_method: ties
base_model: Qwen/Qwen2.5-1.5B
parameters:
normalize: true
dtype: bfloat16