Create README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Muril Large Squad2
|
| 2 |
+
This model is finetuned for QA task on Squad2 from [Muril Large checkpoint](https://huggingface.co/google/muril-large-cased).
|
| 3 |
+
|
| 4 |
+
## Hyperparameters
|
| 5 |
+
```
|
| 6 |
+
Batch Size: 4
|
| 7 |
+
Grad Accumulation Steps = 8
|
| 8 |
+
Total epochs = 3
|
| 9 |
+
MLM Checkpoint = google/muril-large-cased
|
| 10 |
+
max_seq_len = 256
|
| 11 |
+
learning_rate = 1e-5
|
| 12 |
+
lr_schedule = LinearWarmup
|
| 13 |
+
warmup_ratio = 0.1
|
| 14 |
+
doc_stride = 128
|
| 15 |
+
```
|
| 16 |
+
|
| 17 |
+
## Squad 2 Evaluation stats:
|
| 18 |
+
TODO
|
| 19 |
+
|
| 20 |
+
## Limitations
|
| 21 |
+
MuRIL is specifically trained to work on 18 Indic languages and English. This model is not expected to perform well in any other languages. See the MuRIL checkpoint for further details.
|