---
base_model:
- schonsense/70B_Triage
- schonsense/70B_neolithic_rabbit
- schonsense/schonsense_70B_thinkthonk
library_name: transformers
tags:
- mergekit
- merge

---
# smrt

This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).

## Merge Details
### Merge Method

This model was merged using the [SCE](https://arxiv.org/abs/2408.07990) merge method using D:\mergekit\yamls\IPOplectic as a base.

### Models Merged

The following models were included in the merge:
* [schonsense/70B_Triage](https://huggingface.co/schonsense/70B_Triage)
* [schonsense/70B_neolithic_rabbit](https://huggingface.co/schonsense/70B_neolithic_rabbit)
* D:\mergekit\yamls\sce_galaxy_brain
* [schonsense/schonsense_70B_thinkthonk](https://huggingface.co/schonsense/schonsense_70B_thinkthonk)

### Configuration

The following YAML configuration was used to produce this model:

```yaml
merge_method: sce
select_topk: 0.25

models:


  - model: "D:\\mergekit\\yamls\\IPOplectic"
  - model: "D:\\mergekit\\yamls\\sce_galaxy_brain"
  - model: schonsense/70B_Triage
  - model: schonsense/70B_neolithic_rabbit
  - model: schonsense/schonsense_70B_thinkthonk


base_model: "D:\\mergekit\\yamls\\IPOplectic"

parameters:
  normalize: false
  int8_mask: true

dtype: float32
out_dtype: bfloat16

tokenizer:
  source: base
  pad_to_multiple_of: 8

```