| license: apache-2.0 | |
| base_model: | |
| - Qwen/Qwen2.5-Coder-7B | |
| - Qwen/Qwen2.5-Math-7B | |
| tags: | |
| - merge | |
| - mergekit | |
| - dare_ties | |
| - code | |
| - math | |
| pipeline_tag: text-generation | |
| # Qwen2.5-CodeMath-7B 🧮💻 | |
| **Qwen2.5-CodeMath-7B** is a high-capacity fused foundation model created by merging **[`Qwen/Qwen2.5-Coder-7B`](https://huggingface.co/Qwen/Qwen2.5-Coder-7B)** and **[`Qwen/Qwen2.5-Math-7B`](https://huggingface.co/Qwen/Qwen2.5-Math-7B)** using **DARE-TIES**. | |
| ## 📌 Architecture & Merge Configuration | |
| - **Base Anchor**: `Qwen/Qwen2.5-7B` | |
| - **Method**: DARE-TIES (Drop And REscale with Task Vector Interference Elimination) | |
| - **Model 1**: `Qwen/Qwen2.5-Coder-7B` (Weight: `0.60`, Density: `0.80`) | |
| - **Model 2**: `Qwen/Qwen2.5-Math-7B` (Weight: `0.40`, Density: `0.80`) | |
| - **Precision**: `bfloat16` (15.2 GB) | |
| ```yaml | |
| merge_method: dare_ties | |
| base_model: Qwen/Qwen2.5-7B | |
| models: | |
| - model: Qwen/Qwen2.5-Coder-7B | |
| parameters: | |
| weight: 0.6 | |
| density: 0.8 | |
| - model: Qwen/Qwen2.5-Math-7B | |
| parameters: | |
| weight: 0.4 | |
| density: 0.8 | |
| parameters: | |
| int8_mask: true | |
| dtype: bfloat16 | |
| tokenizer_source: Qwen/Qwen2.5-Coder-7B | |
| ``` | |