---
base_model:
- OronoCris/affine-83-5ETkv1LSG1sMVBma1oPQUUKPkzmLDHQN7zwf8sWqCQEps3yR
- vera6/affine-5GmvsbydEDvdhHy6w8JHMCorYyJPiAnvzHWaa8Aj6iTdzzns
- DuWenJing/affine-model-5CrVf11voFhxLs2nM7LBVUKPfD4gNwYugag4Dmwe8JM423RC
library_name: transformers
tags:
- mergekit
- merge
---
# trained_models

This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).

## Merge Details
### Merge Method

This model was merged using the [Model Breadcrumbs with TIES](https://arxiv.org/abs/2312.06795) merge method using [vera6/affine-5GmvsbydEDvdhHy6w8JHMCorYyJPiAnvzHWaa8Aj6iTdzzns](https://huggingface.co/vera6/affine-5GmvsbydEDvdhHy6w8JHMCorYyJPiAnvzHWaa8Aj6iTdzzns) as a base.

### Models Merged

The following models were included in the merge:
* [OronoCris/affine-83-5ETkv1LSG1sMVBma1oPQUUKPkzmLDHQN7zwf8sWqCQEps3yR](https://huggingface.co/OronoCris/affine-83-5ETkv1LSG1sMVBma1oPQUUKPkzmLDHQN7zwf8sWqCQEps3yR)
* [DuWenJing/affine-model-5CrVf11voFhxLs2nM7LBVUKPfD4gNwYugag4Dmwe8JM423RC](https://huggingface.co/DuWenJing/affine-model-5CrVf11voFhxLs2nM7LBVUKPfD4gNwYugag4Dmwe8JM423RC)

### Configuration

The following YAML configuration was used to produce this model:

```yaml
merge_method: breadcrumbs_ties
base_model: vera6/affine-5GmvsbydEDvdhHy6w8JHMCorYyJPiAnvzHWaa8Aj6iTdzzns
models:
  - model: DuWenJing/affine-model-5CrVf11voFhxLs2nM7LBVUKPfD4gNwYugag4Dmwe8JM423RC
    parameters:
      density: 0.95
      weight: 0.7  # Increased weight for better performance
  - model: OronoCris/affine-83-5ETkv1LSG1sMVBma1oPQUUKPkzmLDHQN7zwf8sWqCQEps3yR
    parameters:
      density: 0.8
      weight: 0.3  # Decreased weight for lower performance
parameters:
  density: 0.92
  gamma: 0.02
dtype: bfloat16

```