metadata
base_model:
- mistralai/Mistral-7B-v0.1
- Lambent/threebird-scribe-alpha0.1-7B
- FelixChao/Patronum-7B
library_name: transformers
tags:
- mergekit
- merge
license: apache-2.0
nextbird
This is a merge of pre-trained language models created using mergekit.
| Tasks | Version | Filter | n-shot | Metric | Value | Stderr | ||
|---|---|---|---|---|---|---|---|---|
| eq_bench | 2.1 | none | 0 | eqbench | ↑ | 78.7229 | ± | 1.4751 |
| none | 0 | percent_parseable | ↑ | 100.0000 | ± | 0.0000 |
Slight improvement over both constituents on EQ-Bench benchmark, again rivaling senseable/WestLake-7B-v2 ancestor while remaining able to be parsed.
Lambent/threebird-scribe-alpha0.1-7B is a local mix of fine-tunes of Lambent/threebird-7B aiming just to fix syntax issues with fairly low forgetting or loss of intelligence.
Merge Details
Merge Method
This model was merged using the TIES merge method using mistralai/Mistral-7B-v0.1 as a base.
Models Merged
The following models were included in the merge:
Configuration
The following YAML configuration was used to produce this model:
models:
- model: Lambent/threebird-scribe-alpha0.1-7B
parameters:
density: 1.0
weight: 1.0
- model: FelixChao/Patronum-7B
parameters:
density: 1.0
weight: 1.0
base_model: mistralai/Mistral-7B-v0.1
merge_method: ties
dtype: float16