Create README.md

README.md

@@ -0,0 +1,98 @@
+---
+license: apache-2.0
+tags:
+- token-classification
+datasets:
+- conll2003
+- conllpp
+language:
+- en
+metrics:
+- f1: 92.85
+- f1(valid): 96.71
+- f1(CoNLLpp(2023)): 92.35
+- f1(CoNLLpp(CrossWeigh)): 94.26
+---
+
+
+# Roberta-Base-CoNLL2003
+
+This model is a fine-tuned version of [roberta-base](https://huggingface.co/roberta-base) on the conll2003 dataset.
+
+## Model Usage
+
+We create the original tokenizer with [BPE-Dropout](https://aclanthology.org/2020.acl-main.170/). 
+So, you can't use AutoTokenizer but if subword normalization is not used, original RobertaTokenizer can be substituted.
+
+Example and Tokenizer Repository: [github](https://github.com/4ldk/CoNLL2003_Choices)
+
+```python
+from transformers import RobertaTokenizer, AutoModelForTokenClassification
+from transformers import pipeline
+
+tokenizer = RobertaTokenizer.from_pretrained("4ldk/Roberta-Base-CoNLL2003")
+model = AutoModelForTokenClassification.from_pretrained("4ldk/Roberta-Base-CoNLL2003")
+
+nlp = pipeline("ner", model=model, tokenizer=tokenizer, grouped_entities=True)
+example = "My name is Philipp and live in Germany"
+
+nlp(example)
+
+```
+
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 5e-5
+- train_batch_size: 32
+- eval_batch_size: 32
+- seed: 42
+- optimizer: AdamW with betas=(0.9,0.999), epsilon=1e-08, and weight decay=0.01
+- lr_scheduler_type: linear with warmup rate = 0.1
+- num_epochs: 20
+- subword regularization p = 0.0 (= trained without subword regularization)
+
+And we add the sentences following the input sentence in the original dataset. Therefore, it cannot be reproduced from the dataset published on huggingface.
+For detail in [our github repository](https://github.com/4ldk/CoNLL2003_Choices/blob/develop/src/utils.py)).
+
+### Training results
+
+#### CoNNL2003
+
+It achieves the following results on the evaluation set:
+- Precision: 0.9707
+- Recall: 0.9636
+- F1: 0.9671
+
+It achieves the following results on the test set:
+- Precision: 0.9352 
+- Recall: 0.9218
+- F1: 0.9285
+
+#### CoNNLpp(2023)
+
+[Do CoNLL-2003 Named Entity Taggers Still Work Well in 2023](
+https://aclanthology.org/2023.acl-long.459.pdf)
+([github](https://github.com/ShuhengL/acl2023_conllpp))
+
+- Precision: 0.9244
+- Recall: 0.9225
+- F1: 0.9235
+
+#### CoNLLpp(CrossWeigh)
+
+[CrossWeigh: Training Named Entity Tagger from Imperfect Annotations](https://aclanthology.org/D19-1519/)
+([github](https://github.com/ZihanWangKi/CrossWeigh))
+
+- Precision: 0.9449
+- Recall: 0.9403
+- F1: 0.9426
+
+
+### Framework versions
+
+- Transformers 4.35.2
+- Pytorch 2.0.1+cu117