Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time
Paper • 2203.05482 • Published • 8
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("Bread-AI/Crumb-13B")
model = AutoModelForCausalLM.from_pretrained("Bread-AI/Crumb-13B")Quick Links
Crumb 13B
Merge of Noromaid, Thorns, and WizardLM for Bread AI
Merge Details
Merge Method
This model was merged using the linear merge method.
Models Merged
The following models were included in the merge:
Configuration
The following YAML configuration was used to produce this model:
models:
- model: NeverSleep/Noromaid-13b-v0.1.1
parameters:
weight: 1.0
- model: WizardLM/WizardLM-13B-V1.2
parameters:
weight: 0.3
- model: CalderaAI/13B-Thorns-l2
parameters:
weight: 0.5
merge_method: linear
dtype: float16
- Downloads last month
- 5
Model tree for Bread-AI/Crumb-13B
Merge model
this model
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Bread-AI/Crumb-13B")