shadowml commited on
Commit
fcc5a71
·
1 Parent(s): db42487

Upload folder using huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +34 -0
README.md ADDED
@@ -0,0 +1,34 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ tags:
4
+ - merge
5
+ - mergekit
6
+ ---
7
+
8
+ # Macaron-7B
9
+
10
+ This model is a merge of the following models made with [mergekit](https://github.com/cg123/mergekit):
11
+ * [OpenPipe/mistral-ft-optimized-1218](https://huggingface.co/OpenPipe/mistral-ft-optimized-1218)
12
+ * [mlabonne/NeuralHermes-2.5-Mistral-7B](https://huggingface.co/mlabonne/NeuralHermes-2.5-Mistral-7B)
13
+
14
+ ## 🧩 Configuration
15
+
16
+ ```yaml
17
+ models:
18
+ - model: mistralai/Mistral-7B-v0.1
19
+ # no parameters necessary for base model
20
+ - model: OpenPipe/mistral-ft-optimized-1218
21
+ parameters:
22
+ density: 0.5
23
+ weight: 0.5
24
+ - model: mlabonne/NeuralHermes-2.5-Mistral-7B
25
+ parameters:
26
+ density: 0.5
27
+ weight: 0.3
28
+ merge_method: ties
29
+ base_model: mistralai/Mistral-7B-v0.1
30
+ parameters:
31
+ normalize: true
32
+ int8_mask: true
33
+ dtype: float16
34
+ ```