|
|
--- |
|
|
license: apache-2.0 |
|
|
base_model: bert-base-uncased |
|
|
tags: |
|
|
- generated_from_trainer |
|
|
metrics: |
|
|
- precision |
|
|
- recall |
|
|
- f1 |
|
|
- accuracy |
|
|
model-index: |
|
|
- name: test-ner |
|
|
results: [] |
|
|
datasets: |
|
|
- ktgiahieu/maccrobat2018_2020 |
|
|
--- |
|
|
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
|
|
# test-ner |
|
|
|
|
|
This model is a fine-tuned version of [bert-base-uncased](https://huggingface.co/bert-base-uncased) on ktgiahieu/maccrobat2018_2020 dataset. |
|
|
It achieves the following results on the evaluation set: |
|
|
- Loss: 0.3155 |
|
|
- Precision: 0.9157 |
|
|
- Recall: 0.9323 |
|
|
- F1: 0.9239 |
|
|
- Accuracy: 0.9530 |
|
|
|
|
|
## Model description |
|
|
|
|
|
More information needed |
|
|
|
|
|
## Intended uses & limitations |
|
|
|
|
|
More information needed |
|
|
|
|
|
## Training and evaluation data |
|
|
|
|
|
More information needed |
|
|
|
|
|
## Training procedure |
|
|
|
|
|
### Training hyperparameters |
|
|
|
|
|
The following hyperparameters were used during training: |
|
|
- learning_rate: 2e-05 |
|
|
- train_batch_size: 4 |
|
|
- eval_batch_size: 4 |
|
|
- seed: 42 |
|
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
|
- lr_scheduler_type: linear |
|
|
- num_epochs: 50 |
|
|
|
|
|
### Training results |
|
|
|
|
|
| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1 | Accuracy | |
|
|
|:-------------:|:-----:|:----:|:---------------:|:---------:|:------:|:------:|:--------:| |
|
|
| 0.0316 | 1.25 | 100 | 0.3050 | 0.8799 | 0.9250 | 0.9019 | 0.9466 | |
|
|
| 0.0296 | 2.5 | 200 | 0.2946 | 0.8904 | 0.9242 | 0.9070 | 0.9497 | |
|
|
| 0.0258 | 3.75 | 300 | 0.2883 | 0.9006 | 0.9196 | 0.9100 | 0.9496 | |
|
|
| 0.0227 | 5.0 | 400 | 0.2905 | 0.8888 | 0.9279 | 0.9079 | 0.9496 | |
|
|
| 0.0183 | 6.25 | 500 | 0.2950 | 0.8864 | 0.9319 | 0.9086 | 0.9496 | |
|
|
| 0.0161 | 7.5 | 600 | 0.2931 | 0.8914 | 0.9273 | 0.9090 | 0.9503 | |
|
|
| 0.0153 | 8.75 | 700 | 0.3069 | 0.8947 | 0.9309 | 0.9124 | 0.9502 | |
|
|
| 0.0131 | 10.0 | 800 | 0.3032 | 0.8927 | 0.9261 | 0.9091 | 0.9497 | |
|
|
| 0.0117 | 11.25 | 900 | 0.2935 | 0.9035 | 0.9295 | 0.9163 | 0.9526 | |
|
|
| 0.0111 | 12.5 | 1000 | 0.3117 | 0.8993 | 0.9290 | 0.9139 | 0.9509 | |
|
|
| 0.0089 | 13.75 | 1100 | 0.3247 | 0.8902 | 0.9293 | 0.9094 | 0.9502 | |
|
|
| 0.0115 | 15.0 | 1200 | 0.3121 | 0.8875 | 0.9383 | 0.9122 | 0.9491 | |
|
|
| 0.0088 | 16.25 | 1300 | 0.3038 | 0.9058 | 0.9301 | 0.9178 | 0.9523 | |
|
|
| 0.0083 | 17.5 | 1400 | 0.3237 | 0.8928 | 0.9336 | 0.9127 | 0.9513 | |
|
|
| 0.0095 | 18.75 | 1500 | 0.3223 | 0.8885 | 0.9389 | 0.9130 | 0.9493 | |
|
|
| 0.0078 | 20.0 | 1600 | 0.3335 | 0.8898 | 0.9380 | 0.9133 | 0.9505 | |
|
|
| 0.0088 | 21.25 | 1700 | 0.3004 | 0.9070 | 0.9332 | 0.9199 | 0.9534 | |
|
|
| 0.0068 | 22.5 | 1800 | 0.3424 | 0.8913 | 0.9350 | 0.9126 | 0.9497 | |
|
|
| 0.0065 | 23.75 | 1900 | 0.3150 | 0.9034 | 0.9338 | 0.9184 | 0.9538 | |
|
|
| 0.011 | 25.0 | 2000 | 0.3097 | 0.9044 | 0.9341 | 0.9190 | 0.9523 | |
|
|
| 0.0082 | 26.25 | 2100 | 0.3101 | 0.9057 | 0.9301 | 0.9177 | 0.9527 | |
|
|
| 0.0108 | 27.5 | 2200 | 0.3143 | 0.9083 | 0.9311 | 0.9196 | 0.9525 | |
|
|
| 0.0081 | 28.75 | 2300 | 0.3211 | 0.9011 | 0.9371 | 0.9188 | 0.9525 | |
|
|
| 0.0089 | 30.0 | 2400 | 0.3357 | 0.8996 | 0.9362 | 0.9175 | 0.9509 | |
|
|
| 0.0074 | 31.25 | 2500 | 0.3097 | 0.9079 | 0.9305 | 0.9190 | 0.9517 | |
|
|
| 0.0077 | 32.5 | 2600 | 0.3253 | 0.9032 | 0.9373 | 0.9199 | 0.9511 | |
|
|
| 0.0076 | 33.75 | 2700 | 0.3252 | 0.9056 | 0.9337 | 0.9194 | 0.9526 | |
|
|
| 0.0058 | 35.0 | 2800 | 0.3422 | 0.8981 | 0.9382 | 0.9177 | 0.9512 | |
|
|
| 0.0067 | 36.25 | 2900 | 0.3323 | 0.9074 | 0.9375 | 0.9222 | 0.9532 | |
|
|
| 0.0063 | 37.5 | 3000 | 0.3390 | 0.9066 | 0.9342 | 0.9202 | 0.9521 | |
|
|
| 0.0053 | 38.75 | 3100 | 0.3241 | 0.9095 | 0.9374 | 0.9232 | 0.9537 | |
|
|
| 0.0054 | 40.0 | 3200 | 0.3211 | 0.9017 | 0.9401 | 0.9205 | 0.9534 | |
|
|
| 0.0051 | 41.25 | 3300 | 0.3339 | 0.8931 | 0.9407 | 0.9163 | 0.9500 | |
|
|
| 0.0064 | 42.5 | 3400 | 0.3514 | 0.8977 | 0.9373 | 0.9170 | 0.9517 | |
|
|
| 0.0056 | 43.75 | 3500 | 0.3327 | 0.9069 | 0.9371 | 0.9218 | 0.9528 | |
|
|
| 0.0053 | 45.0 | 3600 | 0.3344 | 0.9034 | 0.9356 | 0.9192 | 0.9525 | |
|
|
| 0.0048 | 46.25 | 3700 | 0.3203 | 0.9171 | 0.9355 | 0.9262 | 0.9542 | |
|
|
| 0.0063 | 47.5 | 3800 | 0.3293 | 0.9109 | 0.9364 | 0.9234 | 0.9530 | |
|
|
| 0.0037 | 48.75 | 3900 | 0.3375 | 0.9146 | 0.9315 | 0.9230 | 0.9520 | |
|
|
| 0.0056 | 50.0 | 4000 | 0.3155 | 0.9157 | 0.9323 | 0.9239 | 0.9530 | |
|
|
|
|
|
|
|
|
|
|
|
### Ner Labels |
|
|
"O", |
|
|
"B-ACTIVITY", |
|
|
"I-ACTIVITY", |
|
|
"I-ADMINISTRATION", |
|
|
"B-ADMINISTRATION", |
|
|
"B-AGE", |
|
|
"I-AGE", |
|
|
"I-AREA", |
|
|
"B-AREA", |
|
|
"B-BIOLOGICAL_ATTRIBUTE", |
|
|
"I-BIOLOGICAL_ATTRIBUTE", |
|
|
"I-BIOLOGICAL_STRUCTURE", |
|
|
"B-BIOLOGICAL_STRUCTURE", |
|
|
"B-CLINICAL_EVENT", |
|
|
"I-CLINICAL_EVENT", |
|
|
"B-COLOR", |
|
|
"I-COLOR", |
|
|
"I-COREFERENCE", |
|
|
"B-COREFERENCE", |
|
|
"B-DATE", |
|
|
"I-DATE", |
|
|
"I-DETAILED_DESCRIPTION", |
|
|
"B-DETAILED_DESCRIPTION", |
|
|
"I-DIAGNOSTIC_PROCEDURE", |
|
|
"B-DIAGNOSTIC_PROCEDURE", |
|
|
"I-DISEASE_DISORDER", |
|
|
"B-DISEASE_DISORDER", |
|
|
"B-DISTANCE", |
|
|
"I-DISTANCE", |
|
|
"B-DOSAGE", |
|
|
"I-DOSAGE", |
|
|
"I-DURATION", |
|
|
"B-DURATION", |
|
|
"I-FAMILY_HISTORY", |
|
|
"B-FAMILY_HISTORY", |
|
|
"B-FREQUENCY", |
|
|
"I-FREQUENCY", |
|
|
"I-HEIGHT", |
|
|
"B-HEIGHT", |
|
|
"B-HISTORY", |
|
|
"I-HISTORY", |
|
|
"I-LAB_VALUE", |
|
|
"B-LAB_VALUE", |
|
|
"I-MASS", |
|
|
"B-MASS", |
|
|
"I-MEDICATION", |
|
|
"B-MEDICATION", |
|
|
"I-NONBIOLOGICAL_LOCATION", |
|
|
"B-NONBIOLOGICAL_LOCATION", |
|
|
"I-OCCUPATION", |
|
|
"B-OCCUPATION", |
|
|
"B-OTHER_ENTITY", |
|
|
"I-OTHER_ENTITY", |
|
|
"B-OTHER_EVENT", |
|
|
"I-OTHER_EVENT", |
|
|
"I-OUTCOME", |
|
|
"B-OUTCOME", |
|
|
"I-PERSONAL_BACKGROUND", |
|
|
"B-PERSONAL_BACKGROUND", |
|
|
"B-QUALITATIVE_CONCEPT", |
|
|
"I-QUALITATIVE_CONCEPT", |
|
|
"I-QUANTITATIVE_CONCEPT", |
|
|
"B-QUANTITATIVE_CONCEPT", |
|
|
"B-SEVERITY", |
|
|
"I-SEVERITY", |
|
|
"B-SEX", |
|
|
"I-SEX", |
|
|
"B-SHAPE", |
|
|
"I-SHAPE", |
|
|
"B-SIGN_SYMPTOM", |
|
|
"I-SIGN_SYMPTOM", |
|
|
"B-SUBJECT", |
|
|
"I-SUBJECT", |
|
|
"B-TEXTURE", |
|
|
"I-TEXTURE", |
|
|
"B-THERAPEUTIC_PROCEDURE", |
|
|
"I-THERAPEUTIC_PROCEDURE", |
|
|
"I-TIME", |
|
|
"B-TIME", |
|
|
"B-VOLUME", |
|
|
"I-VOLUME", |
|
|
"I-WEIGHT", |
|
|
"B-WEIGHT", |
|
|
|
|
|
### Framework versions |
|
|
|
|
|
- Transformers 4.38.2 |
|
|
- Pytorch 2.2.1+cu121 |
|
|
- Datasets 2.18.0 |
|
|
- Tokenizers 0.15.2 |
