Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time
Paper • 2203.05482 • Published • 8
How to use from
SGLangUse Docker images
docker run --gpus all \
--shm-size 32g \
-p 30000:30000 \
-v ~/.cache/huggingface:/root/.cache/huggingface \
--env "HF_TOKEN=<secret>" \
--ipc=host \
lmsysorg/sglang:latest \
python3 -m sglang.launch_server \
--model-path "Nisk36/MergeModelAll" \
--host 0.0.0.0 \
--port 30000# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
-H "Content-Type: application/json" \
--data '{
"model": "Nisk36/MergeModelAll",
"prompt": "Once upon a time,",
"max_tokens": 512,
"temperature": 0.5
}'Quick Links
final_model
This is a merge of pre-trained language models created using mergekit.
Merge Details
Merge Method
This model was merged using the linear merge method.
Models Merged
The following models were included in the merge:
Configuration
The following YAML configuration was used to produce this model:
dtype: bfloat16
merge_method: linear
parameters:
int8_mask: 1.0
normalize: 1.0
slices:
- sources:
- layer_range: [0, 4]
model: Nisk36/finetuned-lmsys_vicuna-7b-v1.5
parameters:
weight: 0.6235769265047518
- layer_range: [0, 4]
model: Nisk36/FT_elyza_ELYZA-japanese-Llama-2-7b-instruct
parameters:
weight: 0.7274442555681364
- sources:
- layer_range: [4, 8]
model: Nisk36/finetuned-lmsys_vicuna-7b-v1.5
parameters:
weight: 0.5271398694239577
- layer_range: [4, 8]
model: Nisk36/FT_elyza_ELYZA-japanese-Llama-2-7b-instruct
parameters:
weight: 0.3489250438855029
- sources:
- layer_range: [8, 12]
model: Nisk36/finetuned-lmsys_vicuna-7b-v1.5
parameters:
weight: 0.15496421762028023
- layer_range: [8, 12]
model: Nisk36/FT_elyza_ELYZA-japanese-Llama-2-7b-instruct
parameters:
weight: 0.541330668871115
- sources:
- layer_range: [12, 16]
model: Nisk36/finetuned-lmsys_vicuna-7b-v1.5
parameters:
weight: 0.5267269624685371
- layer_range: [12, 16]
model: Nisk36/FT_elyza_ELYZA-japanese-Llama-2-7b-instruct
parameters:
weight: 0.8265113027826562
- sources:
- layer_range: [16, 20]
model: Nisk36/finetuned-lmsys_vicuna-7b-v1.5
parameters:
weight: 0.6599861585345389
- layer_range: [16, 20]
model: Nisk36/FT_elyza_ELYZA-japanese-Llama-2-7b-instruct
parameters:
weight: -0.249060520039947
- sources:
- layer_range: [20, 24]
model: Nisk36/finetuned-lmsys_vicuna-7b-v1.5
parameters:
weight: 0.7761318532349375
- layer_range: [20, 24]
model: Nisk36/FT_elyza_ELYZA-japanese-Llama-2-7b-instruct
parameters:
weight: 0.7040995904551324
- sources:
- layer_range: [24, 28]
model: Nisk36/finetuned-lmsys_vicuna-7b-v1.5
parameters:
weight: 0.40152017541360374
- layer_range: [24, 28]
model: Nisk36/FT_elyza_ELYZA-japanese-Llama-2-7b-instruct
parameters:
weight: 0.767141768059921
- sources:
- layer_range: [28, 32]
model: Nisk36/finetuned-lmsys_vicuna-7b-v1.5
parameters:
weight: -0.004536646708608122
- layer_range: [28, 32]
model: Nisk36/FT_elyza_ELYZA-japanese-Llama-2-7b-instruct
parameters:
weight: 0.8295357241419378
- Downloads last month
- 2
Model tree for Nisk36/MergeModelAll
Merge model
this model
Install from pip and serve model
# Install SGLang from pip: pip install sglang# Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Nisk36/MergeModelAll" \ --host 0.0.0.0 \ --port 30000# Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Nisk36/MergeModelAll", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'