Update README.md
Browse files
README.md
CHANGED
|
@@ -51,6 +51,56 @@ Which will generate something like:
|
|
| 51 |
|
| 52 |
The model will automatically emit an end-of-text token (`</s>`) when it judges that the response is complete.
|
| 53 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 54 |
## Limitations and biases
|
| 55 |
|
| 56 |
The intended use-case for this model is fictional conversation for entertainment purposes. Any other sort of usage is out of scope.
|
|
|
|
| 51 |
|
| 52 |
The model will automatically emit an end-of-text token (`</s>`) when it judges that the response is complete.
|
| 53 |
|
| 54 |
+
## Eval / Benchmark scores
|
| 55 |
+
|
| 56 |
+
Current eval out of the current 16bit Pygmalion-13b model: <br>
|
| 57 |
+
<html>
|
| 58 |
+
<head>
|
| 59 |
+
<style>
|
| 60 |
+
table {
|
| 61 |
+
border:1px solid #b3adad;
|
| 62 |
+
border-collapse:collapse;
|
| 63 |
+
padding:5px;
|
| 64 |
+
}
|
| 65 |
+
table th {
|
| 66 |
+
border:1px solid #b3adad;
|
| 67 |
+
padding:5px;
|
| 68 |
+
background: #f0f0f0;
|
| 69 |
+
color: #313030;
|
| 70 |
+
}
|
| 71 |
+
table td {
|
| 72 |
+
border:1px solid #b3adad;
|
| 73 |
+
text-align:center;
|
| 74 |
+
padding:5px;
|
| 75 |
+
background: #ffffff;
|
| 76 |
+
color: #313030;
|
| 77 |
+
}
|
| 78 |
+
</style>
|
| 79 |
+
</head>
|
| 80 |
+
<body>
|
| 81 |
+
<table>
|
| 82 |
+
<thead>
|
| 83 |
+
<tr>
|
| 84 |
+
<th>Model:</th>
|
| 85 |
+
<th>Wikitext2</th>
|
| 86 |
+
<th>Ptb-New</th>
|
| 87 |
+
<th>C4-New</th>
|
| 88 |
+
</tr>
|
| 89 |
+
</thead>
|
| 90 |
+
<tbody>
|
| 91 |
+
<tr>
|
| 92 |
+
<td>Pygmalion 7b - 16bit</td>
|
| 93 |
+
<td>5.710726737976074</td>
|
| 94 |
+
<td>23.633684158325195</td>
|
| 95 |
+
<td>7.6324849128723145</td>
|
| 96 |
+
</tr>
|
| 97 |
+
</tbody>
|
| 98 |
+
</table>
|
| 99 |
+
</body>
|
| 100 |
+
</html>
|
| 101 |
+
<br><b>Thanks to YellowRose#1776 for the numbers.</b>
|
| 102 |
+
<hr>
|
| 103 |
+
|
| 104 |
## Limitations and biases
|
| 105 |
|
| 106 |
The intended use-case for this model is fictional conversation for entertainment purposes. Any other sort of usage is out of scope.
|