aashish1904 commited on
Commit
c53785d
·
verified ·
1 Parent(s): 41491b7

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +46 -5
README.md CHANGED
@@ -5,13 +5,37 @@ license: apache-2.0
5
 
6
  ---
7
 
8
- <img src = "https://lh7-us.googleusercontent.com/docsz/AD_4nXfrlKyH6elkxeyrKw4el9j8V3IOQLsqTVngg19Akt6se1Eq2xaocCEjOmc1w8mq5ENHeYfpzRWjYB8D4mtmMPsiH7QyX_Ii1kEM7bk8eMzO68y9JEuDcoJxJBgbNDzRbTdVXylN9_zjrEposDwsoN7csKiD?key=xt3VSDoCbmTY7o-cwwOFwQ" style="height: 150px; width: 400px;">
9
 
10
  # QuantFactory/Lite-Mistral-150M-v2-Instruct-GGUF
11
  This is quantized version of [OuteAI/Lite-Mistral-150M-v2-Instruct](https://huggingface.co/OuteAI/Lite-Mistral-150M-v2-Instruct) created using llama.cpp
12
 
13
  # Original Model Card
14
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
15
  # Lite-Mistral-150M-v2-Instruct
16
 
17
  This is a Lite series model based on the Mistral architecture, comprising approximately 157 million parameters. <br>
@@ -29,7 +53,8 @@ The model was trained on ~8 billion tokens.
29
 
30
  ## Chat format
31
 
32
- This model is **very sensitive** to the chat template used. Ensure you use the correct template:
 
33
  ```
34
  <s>system
35
  [System message]</s>
@@ -115,43 +140,59 @@ The model shows some promise in understanding context related to simple requests
115
  <table style="text-align: left;">
116
  <tr>
117
  <th>Benchmark</th>
118
- <th>5-shot</th>
119
- <th>0-shot</th>
 
 
120
  </tr>
121
  <tr>
122
  <td>ARC Easy</td>
123
  <td>47.26</td>
 
124
  <td>45.58</td>
 
125
  </tr>
126
  <tr>
127
  <td>BoolQ</td>
128
  <td>43.33</td>
 
129
  <td>44.16</td>
 
130
  </tr>
131
  <tr>
132
  <td>HellaSWAG</td>
133
  <td>28.70</td>
 
134
  <td>28.72</td>
 
135
  </tr>
136
  <tr>
137
  <td>MMLU</td>
138
- <td>26.09</td>
 
139
  <td>25.28</td>
 
140
  </tr>
141
  <tr>
142
  <td>OpenBookQA</td>
143
  <td>16.00</td>
 
144
  <td>18.20</td>
 
145
  </tr>
146
  <tr>
147
  <td>PIQA</td>
148
  <td>62.79</td>
 
149
  <td>62.02</td>
 
150
  </tr>
151
  <tr>
152
  <td>Winogrande</td>
153
  <td>51.30</td>
 
154
  <td>51.78</td>
 
155
  </tr>
156
  </table>
157
 
 
5
 
6
  ---
7
 
8
+ ![](https://lh7-rt.googleusercontent.com/docsz/AD_4nXeiuCm7c8lEwEJuRey9kiVZsRn2W-b4pWlu3-X534V3YmVuVc2ZL-NXg2RkzSOOS2JXGHutDuyyNAUtdJI65jGTo8jT9Y99tMi4H4MqL44Uc5QKG77B0d6-JfIkZHFaUA71-RtjyYZWVIhqsNZcx8-OMaA?key=xt3VSDoCbmTY7o-cwwOFwQ)
9
 
10
  # QuantFactory/Lite-Mistral-150M-v2-Instruct-GGUF
11
  This is quantized version of [OuteAI/Lite-Mistral-150M-v2-Instruct](https://huggingface.co/OuteAI/Lite-Mistral-150M-v2-Instruct) created using llama.cpp
12
 
13
  # Original Model Card
14
 
15
+ <style>
16
+ table {
17
+ border-collapse: collapse;
18
+ width: 100%;
19
+ margin-bottom: 20px;
20
+ }
21
+ th, td {
22
+ border: 1px solid #ddd;
23
+ padding: 8px;
24
+ text-align: center;
25
+ }
26
+ .best {
27
+ font-weight: bold;
28
+ text-decoration: underline;
29
+ }
30
+ </style>
31
+
32
+ <div style="text-align: center; margin: 20px auto; padding: 20px; border: 3px solid #ddd; border-radius: 10px;">
33
+ <h2 style="margin-bottom: 4px; margin-top: 0px;">OuteAI</h2>
34
+ <a href="https://www.outeai.com/" target="_blank" style="margin-right: 10px;">🌎 OuteAI.com</a>
35
+ <a href="https://discord.gg/vyBM87kAmf" target="_blank" style="margin-right: 10px;">🤝 Join our Discord</a>
36
+ <a href="https://x.com/OuteAI" target="_blank">𝕏 @OuteAI</a>
37
+ </div>
38
+
39
  # Lite-Mistral-150M-v2-Instruct
40
 
41
  This is a Lite series model based on the Mistral architecture, comprising approximately 157 million parameters. <br>
 
53
 
54
  ## Chat format
55
 
56
+ > [!IMPORTANT]
57
+ > This model is **sensitive** to the chat template used. Ensure you use the correct template:
58
  ```
59
  <s>system
60
  [System message]</s>
 
140
  <table style="text-align: left;">
141
  <tr>
142
  <th>Benchmark</th>
143
+ <th>5-shot (acc)</th>
144
+ <th>5-shot (acc_norm)</th>
145
+ <th>0-shot (acc)</th>
146
+ <th>0-shot (acc_norm)</th>
147
  </tr>
148
  <tr>
149
  <td>ARC Easy</td>
150
  <td>47.26</td>
151
+ <td>46.30</td>
152
  <td>45.58</td>
153
+ <td>39.52</td>
154
  </tr>
155
  <tr>
156
  <td>BoolQ</td>
157
  <td>43.33</td>
158
+ <td>-</td>
159
  <td>44.16</td>
160
+ <td>-</td>
161
  </tr>
162
  <tr>
163
  <td>HellaSWAG</td>
164
  <td>28.70</td>
165
+ <td>30.73</td>
166
  <td>28.72</td>
167
+ <td>31.01</td>
168
  </tr>
169
  <tr>
170
  <td>MMLU</td>
171
+ <td>-</td>
172
+ <td>-</td>
173
  <td>25.28</td>
174
+ <td>-</td>
175
  </tr>
176
  <tr>
177
  <td>OpenBookQA</td>
178
  <td>16.00</td>
179
+ <td>28.00</td>
180
  <td>18.20</td>
181
+ <td>28.40</td>
182
  </tr>
183
  <tr>
184
  <td>PIQA</td>
185
  <td>62.79</td>
186
+ <td>62.68</td>
187
  <td>62.02</td>
188
+ <td>60.23</td>
189
  </tr>
190
  <tr>
191
  <td>Winogrande</td>
192
  <td>51.30</td>
193
+ <td>-</td>
194
  <td>51.78</td>
195
+ <td>-</td>
196
  </tr>
197
  </table>
198