README.md

metadata

license: apache-2.0
base_model:
  - Qwen/Qwen2.5-Coder-7B
  - Qwen/Qwen2.5-Math-7B
tags:
  - merge
  - mergekit
  - dare_ties
  - code
  - math
pipeline_tag: text-generation

Qwen2.5-CodeMath-7B 🧮💻

Qwen2.5-CodeMath-7B is a high-capacity fused foundation model created by merging Qwen/Qwen2.5-Coder-7B and Qwen/Qwen2.5-Math-7B using DARE-TIES.

📌 Architecture & Merge Configuration

• Base Anchor: Qwen/Qwen2.5-7B
• Method: DARE-TIES (Drop And REscale with Task Vector Interference Elimination)
• Model 1: Qwen/Qwen2.5-Coder-7B (Weight: 0.60, Density: 0.80)
• Model 2: Qwen/Qwen2.5-Math-7B (Weight: 0.40, Density: 0.80)
• Precision: bfloat16 (15.2 GB)

merge_method: dare_ties
base_model: Qwen/Qwen2.5-7B
models:
  - model: Qwen/Qwen2.5-Coder-7B
    parameters:
      weight: 0.6
      density: 0.8
  - model: Qwen/Qwen2.5-Math-7B
    parameters:
      weight: 0.4
      density: 0.8
parameters:
  int8_mask: true
dtype: bfloat16
tokenizer_source: Qwen/Qwen2.5-Coder-7B