virtuous7373 commited on
Commit
fcf981b
·
verified ·
1 Parent(s): 3955c51

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +145 -0
README.md ADDED
@@ -0,0 +1,145 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model:
3
+ - Lambent/Mira-v1.17-27B-Custom-Heretic
4
+ tags:
5
+ - gguf
6
+ - mergekit
7
+ - merge
8
+ - Lambent
9
+ ---
10
+
11
+ <div align="center">
12
+ <h1 style="font-size: 1.2em; font-weight: bold;">
13
+ All credits go to:</h1>
14
+ <h1 style="font-size: 1.2em; font-weight: bold;"> <a href="https://huggingface.co/Lambent">Lambent</a> for the wonderful model series. They did all the work, I just ran a few commands.</h1>
15
+ </div>
16
+ <div align="center">
17
+ <img src="https://upload.wikimedia.org/wikipedia/commons/thumb/a/a2/Demeter_Altemps_Inv8546.jpg/500px-Demeter_Altemps_Inv8546.jpg" width="200" alt="Aphrodite">
18
+ </div>
19
+ <div align="center">
20
+ <h1 style="font-size: 1.23em; font-weight: bold;">
21
+ <a href="https://web.archive.org/web/20251008221112if_/https://uh.edu/~cldue/texts/demeter.html">Homeric Hymn to Demeter</a></h1>
22
+ </h1>
23
+ </div>
24
+ ### 📖 Overview
25
+
26
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/arcee-ai/mergekit. It utilizes custom merge methods.
27
+
28
+ ### The Recipe
29
+
30
+ ϻira Dεmeter| Methods: [Karcher, MagStock, MagVar, CABS]
31
+
32
+ <details>
33
+ <summary><b>View Configuration</b></summary>
34
+
35
+ ```yaml
36
+ # https://huggingface.co/Lambent
37
+ name: Mira-Aphrodite-v19-freckles
38
+
39
+ merge_method: magstock
40
+
41
+ models:
42
+ - model: ./Ingredients/Lambent_Mira-v1.8.1a-27B
43
+ parameters:
44
+ weight: 0.5
45
+ density: 0.5
46
+ epsilon: 0.4
47
+ - model: ./Ingredients/Mira-v14 # karcher
48
+ parameters:
49
+ weight: 0.5
50
+ density: 0.5
51
+
52
+ base_model: ./Ingredients/Lambent_Mira-v1.8.1a-27B
53
+
54
+ dtype: bfloat16
55
+
56
+ parameters:
57
+ rescale: true
58
+ t_factor: 1.0
59
+
60
+ tokenizer:
61
+ source: union
62
+
63
+ ---
64
+ name: Mira-Aphrodite-v31
65
+
66
+ merge_method: magvar
67
+
68
+ base_model: ./Ingredients/Lambent_Mira-v1.21-27B-rlvr
69
+ models:
70
+ - model: ./Ingredients/Lambent_Mira-v1.21-27B-rlvr
71
+ parameters:
72
+ weight: 0.40
73
+ - model: ./Ingredients/Lambent_Mira-v1.20-27B-dpo
74
+ parameters:
75
+ weight: 0.20
76
+ - model: ./Ingredients/Lambent_Mira-v1.19.3c-27B-dpo
77
+ parameters:
78
+ weight: 0.20
79
+ - model: ./Ingredients/Lambent_Mira-v1.19-27B-dpo
80
+ parameters:
81
+ weight: 0.20
82
+
83
+ parameters:
84
+ top_k: 0.45
85
+ eps: 1e-6
86
+ bond: 0.2
87
+
88
+ tokenizer:
89
+ source: base
90
+
91
+ dtype: float32
92
+ out_dtype: bfloat16
93
+
94
+ ---
95
+ name: Mira-Demeter
96
+
97
+ merge_method: cabs
98
+
99
+ base_model: ./Ingredients/Lambent_Mira-v1.21-27B-rlvr/
100
+
101
+ models:
102
+ - model: ./Ingredients/Lambent_Mira-v1.21-27B-rlvr/
103
+ - model: ./Ingredients/Mira-Aphrodite-v31/
104
+ parameters:
105
+ weight: 0.6
106
+ n_val: 16
107
+ m_val: 32
108
+ - model: ./Ingredients/Mira-Aphrodite-v19-freckles/
109
+ parameters:
110
+ weight: 0.4
111
+ n_val: 11
112
+ m_val: 33
113
+
114
+
115
+ default_n_val: 8
116
+ default_m_val: 32
117
+
118
+ pruning_order:
119
+ - ./Ingredients/Mira-Aphrodite-v31/
120
+ - ./Ingredients/Mira-Aphrodite-v19-freckles/
121
+
122
+ # base_model: base
123
+
124
+ dtype: float32
125
+ out_dtype: bfloat16
126
+
127
+ tokenizer:
128
+ source: union
129
+
130
+ ```
131
+
132
+ ## </details>
133
+
134
+ ### A conversation snippet
135
+
136
+ <details>
137
+ <summary><b>View Conversation</b></summary>
138
+
139
+ ```yaml
140
+ #
141
+ ```
142
+ ```
143
+ </details>
144
+ </div>
145
+ ```