|
|
--- |
|
|
license: apache-2.0 |
|
|
tags: |
|
|
- merge |
|
|
- mergekit |
|
|
- lazymergekit |
|
|
- kodonho/Solar-OrcaDPO-Solar-Instruct-SLERP |
|
|
- VAGOsolutions/SauerkrautLM-SOLAR-Instruct |
|
|
--- |
|
|
|
|
|
# SOLAR-10.7B-Instruct-ties |
|
|
|
|
|
SOLAR-10.7B-Instruct-ties is a merge of the following models using [mergekit](https://github.com/cg123/mergekit): |
|
|
* [kodonho/Solar-OrcaDPO-Solar-Instruct-SLERP](https://huggingface.co/kodonho/Solar-OrcaDPO-Solar-Instruct-SLERP) |
|
|
* [VAGOsolutions/SauerkrautLM-SOLAR-Instruct](https://huggingface.co/VAGOsolutions/SauerkrautLM-SOLAR-Instruct) |
|
|
|
|
|
## 🧩 Configuration |
|
|
|
|
|
```yaml |
|
|
models: |
|
|
- model: upstage/SOLAR-10.7B-Instruct-v1.0 |
|
|
# no parameters necessary for base model |
|
|
- model: kodonho/Solar-OrcaDPO-Solar-Instruct-SLERP |
|
|
parameters: |
|
|
density: 0.5 |
|
|
weight: 0.5 |
|
|
- model: VAGOsolutions/SauerkrautLM-SOLAR-Instruct |
|
|
parameters: |
|
|
density: 0.5 |
|
|
weight: 0.3 |
|
|
merge_method: ties |
|
|
base_model: upstage/SOLAR-10.7B-Instruct-v1.0 |
|
|
parameters: |
|
|
normalize: true |
|
|
dtype: float16 |
|
|
``` |
|
|
## 💻 Example Python Code |
|
|
|
|
|
```python |
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline |
|
|
|
|
|
model_name_or_path = "nfaheem/SOLAR-10.7B-Instruct-ties" |
|
|
model = AutoModelForCausalLM.from_pretrained(model_name_or_path, |
|
|
device_map="auto", |
|
|
revision="main") |
|
|
|
|
|
tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, use_fast=True) |
|
|
|
|
|
prompt = "Write a story about llamas" |
|
|
system_message = "You are a story writing assistant" |
|
|
prompt_template=f'''{prompt} |
|
|
''' |
|
|
|
|
|
print("\n\n*** Generate:") |
|
|
|
|
|
input_ids = tokenizer(prompt_template, return_tensors='pt').input_ids.cuda() |
|
|
output = model.generate(inputs=input_ids, temperature=0.7, do_sample=True, top_p=0.95, top_k=40, max_new_tokens=512) |
|
|
print(tokenizer.decode(output[0])) |
|
|
|
|
|
# Inference can also be done using transformers' pipeline |
|
|
|
|
|
print("*** Pipeline:") |
|
|
pipe = pipeline( |
|
|
"text-generation", |
|
|
model=model, |
|
|
tokenizer=tokenizer, |
|
|
max_new_tokens=512, |
|
|
do_sample=True, |
|
|
temperature=0.7, |
|
|
top_p=0.95, |
|
|
top_k=40, |
|
|
repetition_penalty=1.1 |
|
|
) |
|
|
|
|
|
print(pipe(prompt_template)[0]['generated_text']) |
|
|
``` |
|
|
## 📋 Summary Eval: |
|
|
|
|
|
| Average | ARC | HellaSwag | MMLU | TruthfulQA | Winogrande | GSM8K | |
|
|
|---------|-------|-----------|--------|------------|------------|-------| |
|
|
| 74.24 | 70.9 | 88.58 | 66.34 | 71.88 | 83.5 | 64.06 | |
|
|
|
|
|
|
|
|
|
|
|
## 📈 Huggingface Leaderboard |
|
|
|
|
|
|
|
|
 |
|
|
|
|
|
|
|
|
|
|
|
|
