Commit ·
8eaf58c
1
Parent(s): 5a77340
add readme
Browse files
README.md
CHANGED
|
@@ -1,3 +1,64 @@
|
|
| 1 |
---
|
| 2 |
license: mit
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: mit
|
| 3 |
---
|
| 4 |
+
# bertimbau-NER
|
| 5 |
+
This model card aims to simplify the use of the [portuguese Bert, a.k.a, Bertimbau](https://github.com/neuralmind-ai/portuguese-bert) for the Named Entity Recognition task.
|
| 6 |
+
|
| 7 |
+
For this model card the we used the BERT-CRF (selective scenario, 5 classes) model available in the [ner_evalutaion](https://github.com/neuralmind-ai/portuguese-bert/tree/master/ner_evaluation) folder of the original Bertimbau repo.
|
| 8 |
+
|
| 9 |
+
## Usage
|
| 10 |
+
|
| 11 |
+
```{python}
|
| 12 |
+
# Load model directly
|
| 13 |
+
from transformers import AutoTokenizer, AutoModelForTokenClassification
|
| 14 |
+
|
| 15 |
+
tokenizer = AutoTokenizer.from_pretrained("marquesafonso/bertimbau-large-ner")
|
| 16 |
+
model = AutoModelForTokenClassification.from_pretrained("marquesafonso/bertimbau-large-ner")
|
| 17 |
+
|
| 18 |
+
```
|
| 19 |
+
|
| 20 |
+
## Example
|
| 21 |
+
|
| 22 |
+
```{python}
|
| 23 |
+
from transformers import pipeline
|
| 24 |
+
|
| 25 |
+
pipe = pipeline("token-classification", model="marquesafonso/bertimbau-large-ner")
|
| 26 |
+
|
| 27 |
+
sentence = "Acima de Ederson, abaixo de Rúben Dias. É entre os dois jogadores do Manchester City que se vai colocar Gonçalo Ramos no ranking de vendas mais avultadas do Benfica."
|
| 28 |
+
|
| 29 |
+
result = pipe([sentence])
|
| 30 |
+
|
| 31 |
+
print(f"{sentence}\n{result}")
|
| 32 |
+
|
| 33 |
+
# Acima de Ederson, abaixo de Rúben Dias. É entre os dois jogadores do Manchester City que se vai colocar Gonçalo Ramos no ranking de vendas mais avultadas do Benfica.
|
| 34 |
+
# [[{'entity': 'B-PESSOA', 'score': 0.99976975, 'index': 4, 'word': 'Ed', 'start': 9, 'end': 11}, {'entity': 'I-PESSOA', 'score': 0.9941182, 'index': 5, 'word': '##erson', 'start': 11, 'end': 16}, {'entity': 'B-PESSOA', 'score': 0.9998306, 'index': 9, 'word': 'R', 'start': 28, 'end': 29}, {'entity': 'I-PESSOA', 'score': 0.9737293, 'index': 10, 'word': '##ú', 'start': 29, 'end': 30}, {'entity': 'I-PESSOA', 'score': 0.9944133, 'index': 11, 'word': '##ben', 'start': 30, 'end': 33}, {'entity': 'I-PESSOA', 'score': 0.9994117, 'index': 12, 'word': 'Dias', 'start': 34, 'end': 38}, {'entity': 'B-ORGANIZACAO', 'score': 0.94043595, 'index': 20, 'word': 'Manchester', 'start': 69, 'end': 79}, {'entity': 'I-ORGANIZACAO', 'score': 0.9870952, 'index': 21, 'word': 'City', 'start': 80, 'end': 84}, {'entity': 'B-PESSOA', 'score': 0.9997652, 'index': 26, 'word': 'Gonçalo', 'start': 104, 'end': 111}, {'entity': 'I-PESSOA', 'score': 0.9989994, 'index': 27, 'word': 'Ramos', 'start': 112, 'end': 117}, {'entity': 'B-ORGANIZACAO', 'score': 0.9033079, 'index': 37, 'word': 'Benfica', 'start': 157, 'end': 164}]]
|
| 35 |
+
```
|
| 36 |
+
|
| 37 |
+
## Acknowledgements
|
| 38 |
+
|
| 39 |
+
This work is an adaptation of [portuguese Bert, a.k.a, Bertimbau](https://github.com/neuralmind-ai/portuguese-bert). You may check and/or cite their [work](http://arxiv.org/abs/1909.10649):
|
| 40 |
+
|
| 41 |
+
```
|
| 42 |
+
@InProceedings{souza2020bertimbau,
|
| 43 |
+
author="Souza, F{\'a}bio and Nogueira, Rodrigo and Lotufo, Roberto",
|
| 44 |
+
editor="Cerri, Ricardo and Prati, Ronaldo C.",
|
| 45 |
+
title="BERTimbau: Pretrained BERT Models for Brazilian Portuguese",
|
| 46 |
+
booktitle="Intelligent Systems",
|
| 47 |
+
year="2020",
|
| 48 |
+
publisher="Springer International Publishing",
|
| 49 |
+
address="Cham",
|
| 50 |
+
pages="403--417",
|
| 51 |
+
isbn="978-3-030-61377-8"
|
| 52 |
+
}
|
| 53 |
+
|
| 54 |
+
|
| 55 |
+
@article{souza2019portuguese,
|
| 56 |
+
title={Portuguese Named Entity Recognition using BERT-CRF},
|
| 57 |
+
author={Souza, F{\'a}bio and Nogueira, Rodrigo and Lotufo, Roberto},
|
| 58 |
+
journal={arXiv preprint arXiv:1909.10649},
|
| 59 |
+
url={http://arxiv.org/abs/1909.10649},
|
| 60 |
+
year={2019}
|
| 61 |
+
}
|
| 62 |
+
```
|
| 63 |
+
|
| 64 |
+
Note that the authors - Fabio Capuano de Souza, Rodrigo Nogueira, Roberto de Alencar Lotufo - have used an MIT LICENSE for their work.
|