Update README.md
Browse files
README.md
CHANGED
|
@@ -6,4 +6,47 @@ datasets:
|
|
| 6 |
model:
|
| 7 |
- facebook/data2vec-text-base
|
| 8 |
---
|
| 9 |
-
<
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 6 |
model:
|
| 7 |
- facebook/data2vec-text-base
|
| 8 |
---
|
| 9 |
+
<h1>data2vec squad</h1>
|
| 10 |
+
This is a testing fine tuned data2vec model in the squad dataset, any improvements and suggestions are welcome!
|
| 11 |
+
|
| 12 |
+
<h3>Intended use</h3>
|
| 13 |
+
Question Answering
|
| 14 |
+
|
| 15 |
+
<h3>Training results</h3>
|
| 16 |
+
<table>
|
| 17 |
+
<thead>
|
| 18 |
+
<tr>
|
| 19 |
+
<th>Epoch</th>
|
| 20 |
+
<th>Training Loss</th>
|
| 21 |
+
<th>Validation Loss</th>
|
| 22 |
+
</tr>
|
| 23 |
+
</thead>
|
| 24 |
+
<tbody>
|
| 25 |
+
<tr>
|
| 26 |
+
<td>1</td>
|
| 27 |
+
<td><span style="color: rgb(33, 33, 33); font-family: Roboto, Noto, sans-serif; font-size: 14px; font-style: normal; font-weight: 400; text-align: right; background-color: rgb(247, 247, 247);">1.015800</span><br></td>
|
| 28 |
+
<td><span style="color: rgb(33, 33, 33); font-family: Roboto, Noto, sans-serif; font-size: 14px; font-style: normal; font-weight: 400; text-align: right; background-color: rgb(247, 247, 247);">0.997690</span><br></td>
|
| 29 |
+
</tr>
|
| 30 |
+
<tr>
|
| 31 |
+
<td>2</td>
|
| 32 |
+
<td><span style="color: rgb(33, 33, 33); font-family: Roboto, Noto, sans-serif; font-size: 14px; font-style: normal; font-weight: 400; text-align: right;">0.804400</span></td>
|
| 33 |
+
<td><span style="color: rgb(33, 33, 33); font-family: Roboto, Noto, sans-serif; font-size: 14px; font-style: normal; font-weight: 400; text-align: right;">0.950322</span><br></td>
|
| 34 |
+
</tr>
|
| 35 |
+
</tbody>
|
| 36 |
+
</table>
|
| 37 |
+
<h3>Hyperparameters</h3>
|
| 38 |
+
<ul>
|
| 39 |
+
<li>evaluation_strategy="epoch"</li>
|
| 40 |
+
<li>learning_rate=2e-5</li>
|
| 41 |
+
<li>per_device_train_batch_size=15</li>
|
| 42 |
+
<li>per_device_eval_batch_size=15</li>
|
| 43 |
+
<li>num_train_epochs=2</li>
|
| 44 |
+
<li>weight_decay=0.01</li>
|
| 45 |
+
</ul>
|
| 46 |
+
|
| 47 |
+
<h3>Frameworks and libraries used:</h3>
|
| 48 |
+
<ul>
|
| 49 |
+
<li>transformers</li>
|
| 50 |
+
<li>datasets</li>
|
| 51 |
+
<li>evaluate</li>
|
| 52 |
+
</ul>
|