| tags: | |
| - model-merge | |
| - hermite-interpolation | |
| - deepseek | |
| base_model: | |
| - deepseek-ai/deepseek-math-7b-instruct | |
| - jahyungu/deepseek-math-7b-instruct_hendrycks_math | |
| # deepseek-7b-math-hendrycksmath-lambda06 | |
| 2モデルの線形補間マージモデル。 | |
| ## Merge Configuration | |
| | Parameter | Value | | |
| |-----------|-------| | |
| | Model A | `deepseek-ai/deepseek-math-7b-instruct` | | |
| | Model B | `jahyungu/deepseek-math-7b-instruct_hendrycks_math` | | |
| | λ_a | 0.60 | | |
| | λ_b | 0.40 | | |
| | Formula | θ* = 0.60 × θ_a + 0.40 × θ_b | | |
| | dtype | torch.float16 | | |
| ## Tokenizer | |
| Union tokenizer (mergekit-style): vocabularies of both models are merged. | |
| - Union vocab size: 100002 | |
| - Tokens added from Model B: 0 | |
| - Tokens only in Model A: 0 | |
| For tokens missing from a model, the other model's embedding is used as fallback | |
| before linear interpolation. | |
| ## Description | |
| This model was created by linearly interpolating the parameters of two models: | |
| - **Model A** (`deepseek-ai/deepseek-math-7b-instruct`): weight = 0.60 | |
| - **Model B** (`jahyungu/deepseek-math-7b-instruct_hendrycks_math`): weight = 0.40 | |