Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,243 @@
+---
+language: pl
+license: mit
+tags:
+  - ner
+datasets:
+  - clarin-pl/kpwr-ner
+metrics:
+  - f1
+  - accuracy
+  - precision
+  - recall
+widget:
+  - text: "Nazywam się Jan Kowalski i mieszkam we Wrocławiu."
+    example_title: "Example"
+---
+
+# FastPDN
+
+FastPolDeepNer is a model designed for easy use, training and configuration. The forerunner of this project is [PolDeepNer2](https://gitlab.clarin-pl.eu/information-extraction/poldeepner2). The model implements a pipeline consisting of data processing and training using: hydra, pytorch, pytorch-lightning, transformers.
+
+## How to use
+
+Here is how to use this model to get the Named Entities in text:
+
+```python
+from transformers import pipeline
+ner = pipeline('ner', model='clarin-pl/FastPDN')
+
+text = "Nazywam się Jan Kowalski i mieszkam we Wrocławiu."
+ner_results = ner(text)
+for output in ner_results:
+    print(output)
+
+{'entity': 'B-nam_liv_person', 'score': 0.99957544, 'index': 4, 'word': 'Jan</w>', 'start': 12, 'end': 15}
+{'entity': 'I-nam_liv_person', 'score': 0.99963534, 'index': 5, 'word': 'Kowalski</w>', 'start': 16, 'end': 24}
+{'entity': 'B-nam_loc_gpe_city', 'score': 0.998931, 'index': 9, 'word': 'Wrocławiu</w>', 'start': 39, 'end': 48}
+```
+
+Here is how to use this model to get the logits for every token in text:
+
+```python
+from transformers import AutoTokenizer, AutoModelForTokenClassification
+
+tokenizer = AutoTokenizer.from_pretrained("clarin-pl/FastPDN")
+model = AutoModelForTokenClassification.from_pretrained("clarin-pl/FastPDN")
+
+text = "Nazywam się Jan Kowalski i mieszkam we Wrocławiu."
+encoded_input = tokenizer(text, return_tensors='pt')
+output = model(**encoded_input)
+```
+
+### Developing
+
+Model pipeline consists of 2 steps:
+
+- Data processing
+- Training
+- (optional) Share model to Hugginface Hub
+
+#### Config
+
+This project use hydra configuration. Every configuration used in this module
+is placed in `.yaml` files in `config` directory.
+
+This directory has structure:
+
+- prepare_data.yaml - main configuration for the data processing stage
+- train.yaml - main configuration for the training stage
+- share_mode.yaml - main configuraion for sharing model to Huggingface Hub
+- callbacks - contains callbacks for pytorch_lightning trainer
+  - default.yaml
+  - early_stopping.yaml
+  - learning_rate_monitor.yaml
+  - model_checkpoint.yaml
+  - rich_progress_bar.yaml
+- datamodule - contains pytorch_lightning datamodule configuration
+  - pdn.yaml
+- experiment - contains all the configurations of executed experiments
+- hydra - hydra configuration files
+- loggers - contains loggers for trainer
+  - csv.yaml
+  - many_loggers.yaml
+  - tensorboards.yaml
+  - wandb.yaml
+- model - contains model architecture hyperparameters
+  - default.yaml
+  - distiluse.yaml
+  - custom_classification_head.yaml
+  - multilabel.yaml
+- paths - contains paths for IO
+- prepare_data - contains configuration for data processing stage
+  - cen_n82
+  - default
+- trainer - contains trainer configurations
+  - default.yaml
+  - cpu.yaml
+  - gpu.yaml
+
+#### Training
+
+1. Install requirements with poetry
+
+```
+poetry install
+```
+
+2. Use poetry environment in next steps
+
+```
+poetry shell
+```
+
+or
+
+```
+poetry run <command>
+```
+
+3. Prepare dataset
+
+```
+python3 src/prepare_data.py experiment=<experiment-name>
+```
+
+4. Train model
+
+```
+CUDA_VISIBLE_DEVICES=<device-id> python3 src/train.py experiment=<experiment-name>
+```
+
+5. (optional) Share model to Huggingface Hub
+
+```
+python3 src/share_model.py
+```
+
+## Evaluation
+
+Runs trained on `cen_n82` and `kpwr_n82`:
+| name |test/f1|test/pdn2_f1|test/acc|test/precision|test/recall|
+|---------|-------|------------|--------|--------------|-----------|
+|distiluse| 0.53 | 0.61 | 0.95 | 0.55 | 0.54 |
+| herbert | 0.68 | 0.78 | 0.97 | 0.7 | 0.69 |
+
+Runs trained and validated only on `cen_n82`:
+| name |test/f1|test/pdn2_f1|test/acc|test/precision|test/recall|
+|----------------|-------|------------|--------|--------------|-----------|
+| distiluse_cen | 0.58 | 0.7 | 0.96 | 0.6 | 0.59 |
+|herbert_cen_bs32| 0.71 | 0.84 | 0.97 | 0.72 | 0.72 |
+| herbert_cen | 0.72 | 0.84 | 0.97 | 0.73 | 0.73 |
+
+Detailed results for `herbert`:
+| tag | f1 |precision|recall|support|
+|-------------------------|----|---------|------|-------|
+| nam_eve_human_cultural |0.65| 0.53 | 0.83 | 88 |
+| nam_pro_title_document |0.87| 0.82 | 0.92 | 50 |
+| nam_loc_gpe_country |0.82| 0.76 | 0.9 | 258 |
+| nam_oth_www |0.71| 0.85 | 0.61 | 18 |
+| nam_liv_person |0.94| 0.89 | 1.0 | 8 |
+| nam_adj_country |0.44| 0.42 | 0.46 | 94 |
+| nam_org_institution |0.15| 0.16 | 0.14 | 22 |
+| nam_loc_land_continent | 0.5| 0.57 | 0.44 | 9 |
+| nam_org_organization |0.64| 0.59 | 0.71 | 58 |
+| nam_liv_god |0.13| 0.09 | 0.25 | 4 |
+| nam_loc_gpe_city |0.56| 0.51 | 0.62 | 87 |
+| nam_org_company | 0.0| 0.0 | 0.0 | 4 |
+| nam_oth_currency |0.71| 0.86 | 0.6 | 10 |
+| nam_org_group_team |0.87| 0.79 | 0.96 | 106 |
+| nam_fac_road |0.67| 0.67 | 0.67 | 6 |
+| nam_fac_park |0.39| 0.7 | 0.27 | 26 |
+| nam_pro_title_tv |0.17| 1.0 | 0.09 | 11 |
+| nam_loc_gpe_admin3 |0.91| 0.97 | 0.86 | 35 |
+| nam_adj |0.47| 0.5 | 0.44 | 9 |
+| nam_loc_gpe_admin1 |0.92| 0.91 | 0.93 | 1146 |
+| nam_oth_tech | 0.0| 0.0 | 0.0 | 4 |
+| nam_pro_brand |0.93| 0.88 | 1.0 | 14 |
+| nam_fac_goe | 0.1| 0.07 | 0.14 | 7 |
+| nam_eve_human |0.76| 0.73 | 0.78 | 74 |
+| nam_pro_vehicle |0.81| 0.79 | 0.83 | 36 |
+| nam_oth | 0.8| 0.82 | 0.79 | 47 |
+| nam_org_nation |0.85| 0.87 | 0.84 | 516 |
+| nam_pro_media_periodic |0.95| 0.94 | 0.96 | 603 |
+| nam_adj_city |0.43| 0.39 | 0.47 | 19 |
+| nam_oth_position |0.56| 0.54 | 0.58 | 26 |
+| nam_pro_title |0.63| 0.68 | 0.59 | 22 |
+| nam_pro_media_tv |0.29| 0.2 | 0.5 | 2 |
+| nam_fac_system |0.29| 0.2 | 0.5 | 2 |
+| nam_eve_human_holiday | 1.0| 1.0 | 1.0 | 2 |
+| nam_loc_gpe_admin2 |0.83| 0.91 | 0.76 | 51 |
+| nam_adj_person |0.86| 0.75 | 1.0 | 3 |
+| nam_pro_software |0.67| 1.0 | 0.5 | 2 |
+| nam_num_house |0.88| 0.9 | 0.86 | 43 |
+| nam_pro_media_web |0.32| 0.43 | 0.25 | 12 |
+| nam_org_group | 0.5| 0.45 | 0.56 | 9 |
+| nam_loc_hydronym_river |0.67| 0.61 | 0.74 | 19 |
+| nam_liv_animal |0.88| 0.79 | 1.0 | 11 |
+| nam_pro_award | 0.8| 1.0 | 0.67 | 3 |
+| nam_pro |0.82| 0.8 | 0.83 | 243 |
+| nam_org_political_party |0.34| 0.38 | 0.32 | 19 |
+| nam_eve_human_sport |0.65| 0.73 | 0.58 | 19 |
+| nam_pro_title_book |0.94| 0.93 | 0.95 | 149 |
+| nam_org_group_band |0.74| 0.73 | 0.75 | 359 |
+| nam_oth_data_format |0.82| 0.88 | 0.76 | 88 |
+| nam_loc_astronomical |0.75| 0.72 | 0.79 | 341 |
+| nam_loc_hydronym_sea | 0.4| 1.0 | 0.25 | 4 |
+| nam_loc_land_mountain |0.95| 0.96 | 0.95 | 74 |
+| nam_loc_land_island |0.55| 0.52 | 0.59 | 46 |
+| nam_num_phone |0.91| 0.91 | 0.91 | 137 |
+| nam_pro_model_car |0.56| 0.64 | 0.5 | 14 |
+| nam_loc_land_region |0.52| 0.5 | 0.55 | 11 |
+| nam_liv_habitant |0.38| 0.29 | 0.54 | 13 |
+| nam_eve |0.47| 0.38 | 0.61 | 85 |
+| nam_loc_historical_region|0.44| 0.8 | 0.31 | 26 |
+| nam_fac_bridge |0.33| 0.26 | 0.46 | 24 |
+| nam_oth_license |0.65| 0.74 | 0.58 | 24 |
+| nam_pro_media |0.33| 0.32 | 0.35 | 52 |
+| nam_loc_gpe_subdivision | 0.0| 0.0 | 0.0 | 9 |
+| nam_loc_gpe_district |0.84| 0.86 | 0.81 | 108 |
+| nam_loc |0.67| 0.6 | 0.75 | 4 |
+| nam_pro_software_game |0.75| 0.61 | 0.95 | 20 |
+| nam_pro_title_album | 0.6| 0.56 | 0.65 | 52 |
+| nam_loc_country_region |0.81| 0.74 | 0.88 | 26 |
+| nam_pro_title_song |0.52| 0.6 | 0.45 | 111 |
+| nam_org_organization_sub| 0.0| 0.0 | 0.0 | 3 |
+| nam_loc_land | 0.4| 0.31 | 0.56 | 36 |
+| nam_fac_square | 0.5| 0.6 | 0.43 | 7 |
+| nam_loc_hydronym |0.67| 0.56 | 0.82 | 11 |
+| nam_loc_hydronym_lake |0.51| 0.44 | 0.61 | 96 |
+| nam_fac_goe_stop |0.35| 0.3 | 0.43 | 7 |
+| nam_pro_media_radio | 0.0| 0.0 | 0.0 | 2 |
+| nam_pro_title_treaty | 0.3| 0.56 | 0.21 | 24 |
+| nam_loc_hydronym_ocean |0.35| 0.38 | 0.33 | 33 |
+
+To see all the experiments and graphs head over to wandb - https://wandb.ai/clarin-pl/FastPDN
+
+## Authors
+
+- Grupa Wieszcze CLARIN-PL
+
+## Contact
+
+- Norbert Ropiak (norbert.ropiak@pwr.edu.pl)