Resolving Interference When Merging Models
Paper
•
2306.01708
•
Published
•
15
Untitled Model (1)
This is a merge of pre-trained language models created using mergekit.
Merge Details
Merge Method
This model was merged using the TIES merge method using T145/KRONOS-8B-V8 as a base.
Models Merged
The following models were included in the merge:
Configuration
The following YAML configuration was used to produce this model:
base_model: T145/KRONOS-8B-V8
dtype: bfloat16
merge_method: ties
parameters:
density: 1.0
weight: 1.0
slices:
- sources:
- layer_range: [0, 32]
model: T145/KRONOS-8B-V1-P1
parameters:
density: 1.0
weight: 1.0
- layer_range: [0, 32]
model: T145/KRONOS-8B-V8
Open LLM Leaderboard Evaluation Results
Detailed results can be found here! Summarized results can be found here!
| Metric | Value (%) |
|---|---|
| Average | 28.78 |
| IFEval (0-Shot) | 78.56 |
| BBH (3-Shot) | 30.07 |
| MATH Lvl 5 (4-Shot) | 19.03 |
| GPQA (0-shot) | 6.15 |
| MuSR (0-shot) | 8.32 |
| MMLU-PRO (5-shot) | 30.57 |
- Downloads last month
- 6
Model tree for T145/KRONOS-8B-V9
Paper for T145/KRONOS-8B-V9
Evaluation results
- averaged accuracy on IFEval (0-Shot)Open LLM Leaderboard78.560
- normalized accuracy on BBH (3-Shot)test set Open LLM Leaderboard30.070
- exact match on MATH Lvl 5 (4-Shot)test set Open LLM Leaderboard19.030
- acc_norm on GPQA (0-shot)Open LLM Leaderboard6.150
- acc_norm on MuSR (0-shot)Open LLM Leaderboard8.320
- accuracy on MMLU-PRO (5-shot)test set Open LLM Leaderboard30.570