---
base_model: []
library_name: transformers
tags:
- mergekit
- merge

---
# test-model-12

This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).

## Merge Details
### Merge Method

This model was merged using the [SCE](https://arxiv.org/abs/2408.07990) merge method using /workspace/cache/models--bruhzair--ignore-merge-10-nemo/snapshots/bc2bc1ac38a2d05d5d32a31dd24d109bcc37c64c as a base.

### Models Merged

The following models were included in the merge:
* /workspace/cache/models--bruhzair--ignore-merge-1/snapshots/0994ed36502b7c1942553cae165a5813acfc7f4b
* /workspace/cache/models--bruhzair--ignore-merge-21/snapshots/88c4df5f3c979ab849c36fb99b11f15330ac2ce7

### Configuration

The following YAML configuration was used to produce this model:

```yaml
models:
  - model: /workspace/cache/models--bruhzair--ignore-merge-1/snapshots/0994ed36502b7c1942553cae165a5813acfc7f4b
    parameters:
      select_topk: 0.4
  - model: /workspace/cache/models--bruhzair--ignore-merge-21/snapshots/88c4df5f3c979ab849c36fb99b11f15330ac2ce7
    parameters:
      select_topk: 0.3
base_model: /workspace/cache/models--bruhzair--ignore-merge-10-nemo/snapshots/bc2bc1ac38a2d05d5d32a31dd24d109bcc37c64c
merge_method: sce
tokenizer:
  source: base
dtype: float32
out_dtype: bfloat16
```