Update README.md
Browse files
README.md
CHANGED
|
@@ -16,4 +16,39 @@ metrics:
|
|
| 16 |
|
| 17 |
---
|
| 18 |
# AsPOS: Pre-trained model for Assamese POS tagging
|
| 19 |
-
AsPOS is a pre-trained POS tagging model focusing on Assamese language. Stacked embedding (MuRIL + FlairEmbedding) and BiLSTM-CRF model are used train the model. It achieves an F1-score of 74.62% in POS tagging with 41 POS tagset.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 16 |
|
| 17 |
---
|
| 18 |
# AsPOS: Pre-trained model for Assamese POS tagging
|
| 19 |
+
AsPOS is a pre-trained POS tagging model focusing on Assamese language. Stacked embedding (MuRIL + FlairEmbedding) and BiLSTM-CRF model are used to train the model. It achieves an F1-score of 74.62% in POS tagging with 41 POS tagset.
|
| 20 |
+
|
| 21 |
+
|
| 22 |
+
## Annotated Assamese POS tagged dataset
|
| 23 |
+
|
| 24 |
+
The dataset has been annotated by an automatic POS tagger, of which the accuracy is 74.62%. After that, it is manually corrected. The dataset is split into three parts for model training, those are train.txt, dev.txt, and test.txt.
|
| 25 |
+
|
| 26 |
+
## How to run
|
| 27 |
+
|
| 28 |
+
Download the pre-trained model from the link- [AsPOS](https://drive.google.com/file/d/1LAi6cZMyRFWoB6uYIWp3CPtTTnfnOCfx/view?usp=sharing)
|
| 29 |
+
|
| 30 |
+
```
|
| 31 |
+
from flair.models import SequenceTagger
|
| 32 |
+
from flair.data import Sentence, Token
|
| 33 |
+
|
| 34 |
+
# Load the tagger
|
| 35 |
+
|
| 36 |
+
model = SequenceTagger.load('AsPOS.pt')
|
| 37 |
+
|
| 38 |
+
# create example sentence
|
| 39 |
+
sen='ফুকন বসুমতাৰী এজন অধ্য়াপক । তেওঁ বৰ্তমান কোকৰাঝাৰত থাকে ।'
|
| 40 |
+
sentence = Sentence(sen)
|
| 41 |
+
# predict tags and print
|
| 42 |
+
model.predict(sentence)
|
| 43 |
+
print(sentence.to_tagged_string())
|
| 44 |
+
ফুকন <N_NNP> বসুমতাৰী <N_NN> এজন <QT_QTF> অধ্য়াপক <N_NN> । <RD_PUNC> তেওঁ <PR_PRP> বৰ্তমান <RB>
|
| 45 |
+
কোকৰাঝাৰত <N_NNP> থাকে <V_VM> । <RD_PUNC>
|
| 46 |
+
|
| 47 |
+
# create example sentence
|
| 48 |
+
sen='মাতৃভাষাৰ সমান্তৰালকৈ সংস্কৃত, ইংৰাজী ভাষাৰ চৰ্চা অত্যন্ত জৰুৰী ৷'
|
| 49 |
+
sentence = Sentence(sen)
|
| 50 |
+
# predict tags and print
|
| 51 |
+
model.predict(sentence)
|
| 52 |
+
print(sentence.to_tagged_string()
|
| 53 |
+
মাতৃভাষাৰ <N_NN> সমান্তৰালকৈ <N_NN> সংস্কৃত <N_NNP> , <RD_PUNC> ইংৰাজী <N_NNP> ভাষাৰ <N_ANN> চৰ্চা <N_NN> অত্যন্ত <RP_INTF>
|
| 54 |
+
জৰুৰী <N_NN> ৷ <RD_PUNC>
|