| license: apache-2.0 | |
| base_model: | |
| - Qwen/Qwen2.5-Coder-1.5B | |
| - Qwen/Qwen2.5-Math-1.5B | |
| tags: | |
| - merge | |
| - mergekit | |
| - dare_ties | |
| - code | |
| - math | |
| pipeline_tag: text-generation | |
| # Qwen2.5-CodeMath-1.5B 🧮💻 | |
| **Qwen2.5-CodeMath-1.5B** is a fused foundation model created by merging **[`Qwen/Qwen2.5-Coder-1.5B`](https://huggingface.co/Qwen/Qwen2.5-Coder-1.5B)** and **[`Qwen/Qwen2.5-Math-1.5B`](https://huggingface.co/Qwen/Qwen2.5-Math-1.5B)** using **DARE-TIES**. | |
| ## 📌 Architecture & Merge Configuration | |
| - **Base Anchor**: `Qwen/Qwen2.5-1.5B` | |
| - **Method**: DARE-TIES (Drop And REscale with Task Vector Interference Elimination) | |
| - **Model 1**: `Qwen/Qwen2.5-Coder-1.5B` (Weight: `0.60`, Density: `0.80`) | |
| - **Model 2**: `Qwen/Qwen2.5-Math-1.5B` (Weight: `0.40`, Density: `0.80`) | |
| - **Precision**: `bfloat16` | |
| ```yaml | |
| merge_method: dare_ties | |
| base_model: Qwen/Qwen2.5-1.5B | |
| models: | |
| - model: Qwen/Qwen2.5-Coder-1.5B | |
| parameters: | |
| weight: 0.6 | |
| density: 0.8 | |
| - model: Qwen/Qwen2.5-Math-1.5B | |
| parameters: | |
| weight: 0.4 | |
| density: 0.8 | |
| parameters: | |
| int8_mask: true | |
| dtype: bfloat16 | |
| tokenizer_source: Qwen/Qwen2.5-Coder-1.5B | |
| ``` | |