|
|
--- |
|
|
license: mit |
|
|
base_model: microsoft/layoutlm-base-uncased |
|
|
tags: |
|
|
- generated_from_trainer |
|
|
datasets: |
|
|
- funsd |
|
|
model-index: |
|
|
- name: layoutlm-funsd |
|
|
results: [] |
|
|
--- |
|
|
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
|
|
# layoutlm-funsd |
|
|
|
|
|
This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on the funsd dataset. |
|
|
It achieves the following results on the evaluation set: |
|
|
- Loss: 0.7043 |
|
|
- Answer: {'precision': 0.7119021134593994, 'recall': 0.7911001236093943, 'f1': 0.7494145199063232, 'number': 809} |
|
|
- Header: {'precision': 0.37209302325581395, 'recall': 0.40336134453781514, 'f1': 0.3870967741935484, 'number': 119} |
|
|
- Question: {'precision': 0.7965796579657966, 'recall': 0.8309859154929577, 'f1': 0.8134191176470588, 'number': 1065} |
|
|
- Overall Precision: 0.7354 |
|
|
- Overall Recall: 0.7893 |
|
|
- Overall F1: 0.7614 |
|
|
- Overall Accuracy: 0.8036 |
|
|
|
|
|
## Model description |
|
|
|
|
|
More information needed |
|
|
|
|
|
## Intended uses & limitations |
|
|
|
|
|
More information needed |
|
|
|
|
|
## Training and evaluation data |
|
|
|
|
|
More information needed |
|
|
|
|
|
## Training procedure |
|
|
|
|
|
### Training hyperparameters |
|
|
|
|
|
The following hyperparameters were used during training: |
|
|
- learning_rate: 3e-05 |
|
|
- train_batch_size: 4 |
|
|
- eval_batch_size: 4 |
|
|
- seed: 42 |
|
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
|
- lr_scheduler_type: linear |
|
|
- num_epochs: 10 |
|
|
- mixed_precision_training: Native AMP |
|
|
|
|
|
### Training results |
|
|
|
|
|
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy | |
|
|
|:-------------:|:-----:|:----:|:---------------:|:--------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------:|:---------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:| |
|
|
| 1.5459 | 1.0 | 38 | 1.0013 | {'precision': 0.4444444444444444, 'recall': 0.5735475896168108, 'f1': 0.500809498111171, 'number': 809} | {'precision': 0.05263157894736842, 'recall': 0.008403361344537815, 'f1': 0.014492753623188406, 'number': 119} | {'precision': 0.5643024162120032, 'recall': 0.67981220657277, 'f1': 0.616695059625213, 'number': 1065} | 0.5068 | 0.5966 | 0.5481 | 0.6648 | |
|
|
| 0.8508 | 2.0 | 76 | 0.7119 | {'precision': 0.6081081081081081, 'recall': 0.7787391841779975, 'f1': 0.6829268292682927, 'number': 809} | {'precision': 0.16455696202531644, 'recall': 0.1092436974789916, 'f1': 0.1313131313131313, 'number': 119} | {'precision': 0.6774703557312253, 'recall': 0.8046948356807512, 'f1': 0.7356223175965665, 'number': 1065} | 0.6303 | 0.7526 | 0.6860 | 0.7691 | |
|
|
| 0.6131 | 3.0 | 114 | 0.6432 | {'precision': 0.6631689401888772, 'recall': 0.7812113720642769, 'f1': 0.717366628830874, 'number': 809} | {'precision': 0.248, 'recall': 0.2605042016806723, 'f1': 0.2540983606557377, 'number': 119} | {'precision': 0.7474048442906575, 'recall': 0.8112676056338028, 'f1': 0.7780279153534444, 'number': 1065} | 0.6835 | 0.7662 | 0.7225 | 0.7837 | |
|
|
| 0.4734 | 4.0 | 152 | 0.6196 | {'precision': 0.6882845188284519, 'recall': 0.8133498145859085, 'f1': 0.7456090651558074, 'number': 809} | {'precision': 0.2569444444444444, 'recall': 0.31092436974789917, 'f1': 0.28136882129277563, 'number': 119} | {'precision': 0.763716814159292, 'recall': 0.8103286384976526, 'f1': 0.7863325740318907, 'number': 1065} | 0.6987 | 0.7817 | 0.7379 | 0.8005 | |
|
|
| 0.3721 | 5.0 | 190 | 0.6197 | {'precision': 0.6894343649946638, 'recall': 0.7985166872682324, 'f1': 0.7399770904925544, 'number': 809} | {'precision': 0.31007751937984496, 'recall': 0.33613445378151263, 'f1': 0.3225806451612903, 'number': 119} | {'precision': 0.7683566433566433, 'recall': 0.8253521126760563, 'f1': 0.7958352195563604, 'number': 1065} | 0.7081 | 0.7852 | 0.7447 | 0.8005 | |
|
|
| 0.2989 | 6.0 | 228 | 0.6483 | {'precision': 0.6992316136114161, 'recall': 0.7873918417799752, 'f1': 0.7406976744186047, 'number': 809} | {'precision': 0.35766423357664234, 'recall': 0.4117647058823529, 'f1': 0.3828125, 'number': 119} | {'precision': 0.7854578096947935, 'recall': 0.8215962441314554, 'f1': 0.8031206975676914, 'number': 1065} | 0.7220 | 0.7832 | 0.7514 | 0.7987 | |
|
|
| 0.2437 | 7.0 | 266 | 0.6707 | {'precision': 0.7067415730337079, 'recall': 0.7775030902348579, 'f1': 0.7404355503237198, 'number': 809} | {'precision': 0.34057971014492755, 'recall': 0.3949579831932773, 'f1': 0.36575875486381326, 'number': 119} | {'precision': 0.7804232804232805, 'recall': 0.8309859154929577, 'f1': 0.8049113233287858, 'number': 1065} | 0.7220 | 0.7832 | 0.7514 | 0.7993 | |
|
|
| 0.2008 | 8.0 | 304 | 0.6904 | {'precision': 0.7038251366120218, 'recall': 0.796044499381953, 'f1': 0.7470997679814385, 'number': 809} | {'precision': 0.3356643356643357, 'recall': 0.40336134453781514, 'f1': 0.366412213740458, 'number': 119} | {'precision': 0.7885304659498208, 'recall': 0.8262910798122066, 'f1': 0.8069692801467218, 'number': 1065} | 0.7231 | 0.7888 | 0.7545 | 0.7990 | |
|
|
| 0.1802 | 9.0 | 342 | 0.7072 | {'precision': 0.7161862527716186, 'recall': 0.7985166872682324, 'f1': 0.7551139684395091, 'number': 809} | {'precision': 0.34459459459459457, 'recall': 0.42857142857142855, 'f1': 0.38202247191011235, 'number': 119} | {'precision': 0.7896174863387978, 'recall': 0.8140845070422535, 'f1': 0.8016643550624134, 'number': 1065} | 0.7281 | 0.7847 | 0.7554 | 0.7989 | |
|
|
| 0.1681 | 10.0 | 380 | 0.7043 | {'precision': 0.7119021134593994, 'recall': 0.7911001236093943, 'f1': 0.7494145199063232, 'number': 809} | {'precision': 0.37209302325581395, 'recall': 0.40336134453781514, 'f1': 0.3870967741935484, 'number': 119} | {'precision': 0.7965796579657966, 'recall': 0.8309859154929577, 'f1': 0.8134191176470588, 'number': 1065} | 0.7354 | 0.7893 | 0.7614 | 0.8036 | |
|
|
|
|
|
|
|
|
### Framework versions |
|
|
|
|
|
- Transformers 4.40.2 |
|
|
- Pytorch 2.3.0+cu121 |
|
|
- Datasets 2.19.1 |
|
|
- Tokenizers 0.19.1 |
|
|
|
