bruhzair/Mutant-Writer-t2-123b

Mutant-t1

This is a merge of pre-trained language models created using mergekit.

Merge Details

Merge Method

This model was merged using the NuSLERP merge method.

Models Merged

The following models were included in the merge:

• /workspace/cache/models--gghfez--Writer-Large-2411-v2.1/snapshots/cb12d12db43e04ae42ac7a3c3ac987c867fd4757
• /workspace/cache/models--TheDrummer--Behemoth-123B-v1.2/snapshots/51354019a02b742aa5a73fe16800225ff719c46d

Configuration

The following YAML configuration was used to produce this model:

dtype: bfloat16
merge_method: nuslerp
slices:
- sources:
  - layer_range: [0, 88]
    model: /workspace/cache/models--TheDrummer--Behemoth-123B-v1.2/snapshots/51354019a02b742aa5a73fe16800225ff719c46d
    parameters:
      weight: [0.75, 0.5, 0.35, 0.1, 0.35, 0.5, 0.75]
  - layer_range: [0, 88]
    model: /workspace/cache/models--gghfez--Writer-Large-2411-v2.1/snapshots/cb12d12db43e04ae42ac7a3c3ac987c867fd4757
    parameters:
      weight: [0.25, 0.5, 0.65, 0.9, 0.65, 0.5, 0.25]