Update readme
Browse files
README.md
CHANGED
|
@@ -70,3 +70,10 @@ labels_predicted = logits.argmax(-1)
|
|
| 70 |
lang_tag_predicted = [model_best.config.id2label[t.item()] for t in labels_predicted[0]]
|
| 71 |
lang_tag_predicted
|
| 72 |
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 70 |
lang_tag_predicted = [model_best.config.id2label[t.item()] for t in labels_predicted[0]]
|
| 71 |
lang_tag_predicted
|
| 72 |
```
|
| 73 |
+
|
| 74 |
+
## Limitations
|
| 75 |
+
The model might show some contradictory or conflicting behavior sometimes. Some of the known (till now) issues are:
|
| 76 |
+
* The model might not be able to predict a small number (typically 1 or 2) of tokens from another language if they are found in the sequence of one language.
|
| 77 |
+
* The model might consider the language of the tokens in a noun phrase the same as the other tokens, even if the language of the tokens (noun phrase) is different.
|
| 78 |
+
* Proper nouns, and some cross-lingual tokens (in, me, etc.) might be wrongly predicted.
|
| 79 |
+
* The prediction also depends on punctuation.
|