| license: apache-2.0 | |
| tags: | |
| - merge | |
| - mergekit | |
| - lazymergekit | |
| - yam-peleg/Experiment26-7B | |
| - chihoonlee10/T3Q-Mistral-Orca-Math-DPO | |
| # trail | |
| trail is a merge of the following models using [mergekit](https://github.com/cg123/mergekit): | |
| * [](https://huggingface.co/) | |
| * [](https://huggingface.co/) | |
| ## 🧩 Configuration | |
| ```json{ | |
| "models": [ | |
| { | |
| "model": "yam-peleg/Experiment26-7B", | |
| "parameters": {} | |
| }, | |
| { | |
| "model": "chihoonlee10/T3Q-Mistral-Orca-Math-DPO", | |
| "parameters": { | |
| "density": 0.53, | |
| "weight": 0.6 | |
| } | |
| } | |
| ], | |
| "merge_method": "dare_ties", | |
| "base_model": "yam-peleg/Experiment26-7B", | |
| "parameters": { | |
| "int8_mask": true, | |
| "dtype": "bfloat16", | |
| "random_seed": 0 | |
| } | |
| } |