Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time
Paper • 2203.05482 • Published • 8
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("MrRobotoAI/X2")
model = AutoModelForCausalLM.from_pretrained("MrRobotoAI/X2")Quick Links
merge
This is a merge of pre-trained language models created using mergekit.
Merge Details
Merge Method
This model was merged using the Linear merge method.
Models Merged
The following models were included in the merge:
Configuration
The following YAML configuration was used to produce this model:
models:
- model: MrRobotoAI/X1+Samhita-kolluri/mistral-paper-critique-lora
- model: MrRobotoAI/Huldra-R1-v2.2-8b-DEEPSEEK
parameters:
weight: 1.0
merge_method: linear
normalize: true
dtype: float16
- Downloads last month
- 5
Model tree for MrRobotoAI/X2
Merge model
this model
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="MrRobotoAI/X2")