README.md

metadata

license: apache-2.0
base_model:
  - Qwen/Qwen2.5-Coder-1.5B
  - Qwen/Qwen2.5-Math-1.5B
tags:
  - merge
  - mergekit
  - dare_ties
  - code
  - math
pipeline_tag: text-generation

Qwen2.5-CodeMath-1.5B 🧮💻

Qwen2.5-CodeMath-1.5B is a fused foundation model created by merging Qwen/Qwen2.5-Coder-1.5B and Qwen/Qwen2.5-Math-1.5B using DARE-TIES.

📌 Architecture & Merge Configuration

• Base Anchor: Qwen/Qwen2.5-1.5B
• Method: DARE-TIES (Drop And REscale with Task Vector Interference Elimination)
• Model 1: Qwen/Qwen2.5-Coder-1.5B (Weight: 0.60, Density: 0.80)
• Model 2: Qwen/Qwen2.5-Math-1.5B (Weight: 0.40, Density: 0.80)
• Precision: bfloat16

merge_method: dare_ties
base_model: Qwen/Qwen2.5-1.5B
models:
  - model: Qwen/Qwen2.5-Coder-1.5B
    parameters:
      weight: 0.6
      density: 0.8
  - model: Qwen/Qwen2.5-Math-1.5B
    parameters:
      weight: 0.4
      density: 0.8
parameters:
  int8_mask: true
dtype: bfloat16
tokenizer_source: Qwen/Qwen2.5-Coder-1.5B