Add model card for NER model

named_entity_recognition_modelcard.md

@@ -0,0 +1,155 @@
+---
+license: mit
+---
+
+# Model Card for named_entity_recognition.pt
+
+This is a fine-tuned model checkpoint for the named entity recognition (NER) task used in the biodata resource inventory performed by the
+[Global Biodata Coalition](https://globalbiodata.org/) in collaboration with [Chan Zuckerberg Initiative](https://chanzuckerberg.com/).
+
+# Model Details
+
+## Model Description
+
+This model has been fine-tuned to detect resource names in scientific articles (title and abstract). This is done using a token classification which assigns predicted
+token labels following the [BIO scheme](https://en.wikipedia.org/wiki/Inside%E2%80%93outside%E2%80%93beginning_(tagging)). These are post-processed to determine the
+predicted "common names" (often an acronym) and "full names" of a resource present in an article.
+
+
+
+- **Developed by:** Ana-Maria Istrate and Kenneth E. Schackart III
+- **Shared by:** Kenneth E. Schackart III
+- **Model type:** RoBERTa (BERT; Transformer)
+- **Language(s) (NLP):** Python
+- **License:** MIT
+- **Finetuned from model:** https://huggingface.co/allenai/dsp_roberta_base_dapt_biomed_tapt_rct_500
+
+## Model Sources
+
+- **Repository:** https://github.com/globalbiodata/inventory_2022/tree/inventory_2022_dev
+- **Paper [optional]:** TBA
+- **Demo [optional]:** TBA
+
+# Uses
+
+This model can be used find predicted biodata resource names in an article's title and abstract
+
+## Direct Use
+
+Direct use of the model has not been assessed or designed.
+
+## Out-of-Scope Use
+
+Model should not be used for anything other than the use described in [uses](named_entity_recognition_modelcard.md#uses).
+
+# Bias, Risks, and Limitations
+
+Biases may have been introduced at several stages of the development and training of this model. First, the model was trained on biomedical corpora
+as described in [Gururangan S., *et al.,* 2020](http://arxiv.org/abs/2004.10964). Second, The model was fine-tuned on scientific articles that were
+manually annotated by 2 curators. Biases in the manual annotation may have affected model fine-tuning. Additionally, manually annotated data were
+procured using a specific search query to Europe PMC, so generalizability may be limited when applying to articles from other sources.
+
+## Recommendations
+
+The model should only be used for identifying resource names in articles from Europe PMC using the
+[query](https://github.com/globalbiodata/inventory_2022/blob/inventory_2022_dev/config/query.txt) present in the GitHub repository.
+Additionally, only article predicted or known to describe a biodata resource should be used.
+
+## How to Get Started with the Model
+
+Follow the direction in the [GitHub repository](https://github.com/globalbiodata/inventory_2022/tree/inventory_2022_dev).
+
+# Training Details
+
+## Training Data
+
+The model was trained on the training split from the [labeled training data](https://github.com/globalbiodata/inventory_2022/blob/inventory_2022_dev/data/manual_ner_extraction.csv).
+
+*Note*: The data can be split into consistent training, validation, testing splits using the procedures detailed in the GitHub repository.
+
+## Training Procedure
+
+The model was trained for 10 epochs, and *F*1-score, precision, recall, and loss were computed after each epoch. The model checkpoint with the highest *F*1-score on the validation
+set was saved (regardless of epoch number).
+
+### Preprocessing
+
+To generate the input to the model, the article title and abstracts were concatenated, separating with one white space character, into a contiguous string. All
+XML tags were removed using a regular expression.
+
+### Speeds, Sizes, Times
+
+The model checkpoint is 496 MB. Speed has not been benchmarked.
+
+# Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+## Testing Data, Factors & Metrics
+
+### Testing Data
+
+<!-- This should link to a Data Card if possible. -->
+
+The model was evaluated using the test split of the [labeled data](https://github.com/globalbiodata/inventory_2022/blob/inventory_2022_dev/data/manual_ner_extraction.csv).
+
+### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+The model was evaluated using *F*1-score, precision, and recall. Precision was prioritized during fine-tuning and model selection.
+
+## Results
+
+- *F*1-score: 0.717
+- Precision: 0.689
+- Recall: 0.748
+
+### Summary
+
+
+
+# Model Examination
+
+The model works satisfactorily for identifying resource names from articles describing biodata resources in the literature.
+
+## Model Architecture and Objective
+
+The base model architecture is as described in [Gururangan S., *et al.,* 2020](http://arxiv.org/abs/2004.10964). Token classification is performed using
+a linear sequence classification layer initialized using [transformers.AutoModelForTokenClassification()](https://huggingface.co/docs/transformers/model_doc/auto).
+
+## Compute Infrastructure
+
+Model was fine-tuned on Google Colaboratory.
+
+### Hardware
+
+Model was fine-tuned using GPU acceleration provided by Google Colaboratory.
+
+### Software
+
+Training software was written in Python.
+
+# Citation
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+TBA
+
+**BibTeX:**
+
+TBA
+
+**APA:**
+
+TBA
+
+# Model Card Authors
+
+This model card was written by Kenneth E. Schackart III.
+
+# Model Card Contact
+
+Ken Schackart: <schackartk1@gmail.com>
+
+