Update README.md

README.md

@@ -15,6 +15,49 @@ pipeline_tag: text-generation
 **Training Corpus**
 - [Intel/orca_dpo_pairs](https://huggingface.co/datasets/Intel/orca_dpo_pairs)
 
+**Explanation**
+- Merge two base models using [mergekit](https://github.com/arcee-ai/mergekit) (slerp)
+- Apply DPO to the merged model, just an adapter part is saved
+- merge the adpater and the merged model
+
+**Merge Script**
+```
+slices:
+  - sources:
+      - model: upstage/SOLAR-10.7B-Instruct-v1.0
+        layer_range: [0, 48]
+      - model: beomi/OPEN-SOLAR-KO-10.7B
+        layer_range: [0, 48]
+merge_method: slerp
+base_model: upstage/SOLAR-10.7B-Instruct-v1.0
+parameters:
+  t:
+    - filter: self_attn
+      value: [0, 0.5, 0.3, 0.7, 1]
+    - filter: mlp
+      value: [1, 0.5, 0.7, 0.3, 0]
+    - value: 0.5 # fallback for rest of tensors
+dtype: float16
+```
+
+**Score**
+|Average|ARC|HellaSwag|MMLU|TruthfulQA|Winogrande|GSM8K|
+|:---:|:---:|:---:|:---:|:---:|:---:|:---:|
+|65.96|63.91|84.58|63.18|51.49|82|50.57|
+
+**Usage**
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
+model = AutoModelForCausalLM.from_pretrained(
+        "dddsaty/SOLAR_Merge_Adapter_DPO_Orca",
+        low_cpu_mem_usage = True,
+        torch_dtype = torch.float16,
+        device_map = device_map,
+    )
+
+tokenizer = AutoTokenizer.from_pretrained("dddsaty/SOLAR_Merge_Adapter_DPO_Orca")
+```
+
 **Log**
 - 2024.02.05: Initial version Upload