mstyslavity/X-prMnt-L_2

merge

This is a merge of pre-trained language models created using mergekit.

Merge Details

Merge Method

This model was merged using the Model Stock merge method using google-bert/bert-large-uncased as a base.

Models Merged

The following models were included in the merge:

• klue/roberta-large
• cointegrated/roberta-large-cola-krishna2020
• OpenAssistant/reward-model-deberta-v3-large-v2
• jiayi1129/spanbert-large-squad
• google-bert/bert-large-cased
• albert/albert-large-v2

Configuration

The following YAML configuration was used to produce this model:

out_dtype: float32
merge_method: model_stock
base_model: google-bert/bert-large-uncased
models:
  - model: cointegrated/roberta-large-cola-krishna2020
    parameters:
      weight: 0.42
  - model: albert/albert-large-v2
    parameters:
      weight: 0.35
  - model: OpenAssistant/reward-model-deberta-v3-large-v2
    parameters:
      weight: 1.0
  - model: google-bert/bert-large-cased
    parameters: 
      weight: 0.24
  - model: klue/roberta-large
    parameters:
      weight: 0.32
  - model: jiayi1129/spanbert-large-squad
    parameters:
      weight: 1.1
parameters:
  normalize: true 
  int8_mask: false