Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time
Paper • 2203.05482 • Published • 8
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("raincandy-u/Test-7B")
model = AutoModelForCausalLM.from_pretrained("raincandy-u/Test-7B")Quick Links
Test-7B
This is a merge of pre-trained language models created using mergekit.
Merge Details
Merge Method
This model was merged using the linear merge method.
Models Merged
The following models were included in the merge:
- E:\UNA-TheBeagle-7b-v1
- E:\go-bruins-v2.1.1
Configuration
The following YAML configuration was used to produce this model:
dtype: float16
merge_method: linear
slices:
- sources:
- layer_range: [0, 32]
model: E:\go-bruins-v2.1.1
parameters:
weight: 1.0
- layer_range: [0, 32]
model: E:\UNA-TheBeagle-7b-v1
parameters:
weight: 1.0
- Downloads last month
- 58
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="raincandy-u/Test-7B")