File size: 9,289 Bytes
f50f16f e03d099 f50f16f e03d099 f50f16f | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 | ---
tags:
- generated_from_trainer
datasets:
- funsd
model-index:
- name: layoutlm-funsd
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# layoutlm-funsd
This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on the funsd dataset.
It achieves the following results on the evaluation set:
- Loss: 0.6847
- Answer: {'precision': 0.7144432194046306, 'recall': 0.8009888751545118, 'f1': 0.7552447552447553, 'number': 809}
- Header: {'precision': 0.30952380952380953, 'recall': 0.3277310924369748, 'f1': 0.31836734693877555, 'number': 119}
- Question: {'precision': 0.7912966252220248, 'recall': 0.8366197183098592, 'f1': 0.81332724783204, 'number': 1065}
- Overall Precision: 0.7309
- Overall Recall: 0.7918
- Overall F1: 0.7601
- Overall Accuracy: 0.8132
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 15
### Training results
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:--------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:|
| 1.8057 | 1.0 | 10 | 1.5966 | {'precision': 0.008733624454148471, 'recall': 0.009888751545117428, 'f1': 0.009275362318840578, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.14909090909090908, 'recall': 0.11549295774647887, 'f1': 0.13015873015873014, 'number': 1065} | 0.0752 | 0.0657 | 0.0702 | 0.3764 |
| 1.4635 | 2.0 | 20 | 1.2374 | {'precision': 0.14137483787289234, 'recall': 0.13473423980222496, 'f1': 0.1379746835443038, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.42204995693367786, 'recall': 0.460093896713615, 'f1': 0.440251572327044, 'number': 1065} | 0.3100 | 0.3006 | 0.3052 | 0.6035 |
| 1.1031 | 3.0 | 30 | 0.9623 | {'precision': 0.4551451187335092, 'recall': 0.4264524103831891, 'f1': 0.44033184428844924, 'number': 809} | {'precision': 0.13157894736842105, 'recall': 0.04201680672268908, 'f1': 0.06369426751592357, 'number': 119} | {'precision': 0.630297565374211, 'recall': 0.6563380281690141, 'f1': 0.6430542778288868, 'number': 1065} | 0.5507 | 0.5263 | 0.5382 | 0.7016 |
| 0.8514 | 4.0 | 40 | 0.7967 | {'precision': 0.6146682188591386, 'recall': 0.6526576019777504, 'f1': 0.6330935251798562, 'number': 809} | {'precision': 0.23333333333333334, 'recall': 0.11764705882352941, 'f1': 0.1564245810055866, 'number': 119} | {'precision': 0.6810422282120395, 'recall': 0.711737089201878, 'f1': 0.6960514233241506, 'number': 1065} | 0.6398 | 0.6523 | 0.6460 | 0.7495 |
| 0.6854 | 5.0 | 50 | 0.7228 | {'precision': 0.6617647058823529, 'recall': 0.723114956736712, 'f1': 0.6910809214412286, 'number': 809} | {'precision': 0.25806451612903225, 'recall': 0.20168067226890757, 'f1': 0.22641509433962265, 'number': 119} | {'precision': 0.697751873438801, 'recall': 0.7868544600938967, 'f1': 0.7396293027360988, 'number': 1065} | 0.6644 | 0.7260 | 0.6938 | 0.7818 |
| 0.5608 | 6.0 | 60 | 0.6733 | {'precision': 0.6585879873551106, 'recall': 0.7725587144622992, 'f1': 0.7110352673492606, 'number': 809} | {'precision': 0.25, 'recall': 0.17647058823529413, 'f1': 0.20689655172413793, 'number': 119} | {'precision': 0.7112561174551386, 'recall': 0.8187793427230047, 'f1': 0.7612396333478829, 'number': 1065} | 0.6720 | 0.7617 | 0.7140 | 0.7976 |
| 0.486 | 7.0 | 70 | 0.6683 | {'precision': 0.670514165792235, 'recall': 0.7898640296662547, 'f1': 0.7253121452894438, 'number': 809} | {'precision': 0.25688073394495414, 'recall': 0.23529411764705882, 'f1': 0.24561403508771928, 'number': 119} | {'precision': 0.7351398601398601, 'recall': 0.7896713615023474, 'f1': 0.7614305115436849, 'number': 1065} | 0.6836 | 0.7566 | 0.7183 | 0.7992 |
| 0.4391 | 8.0 | 80 | 0.6590 | {'precision': 0.6809623430962343, 'recall': 0.8046971569839307, 'f1': 0.7376770538243627, 'number': 809} | {'precision': 0.2641509433962264, 'recall': 0.23529411764705882, 'f1': 0.24888888888888888, 'number': 119} | {'precision': 0.7585324232081911, 'recall': 0.8347417840375587, 'f1': 0.7948144836835047, 'number': 1065} | 0.7019 | 0.7868 | 0.7419 | 0.8042 |
| 0.3834 | 9.0 | 90 | 0.6569 | {'precision': 0.7043189368770764, 'recall': 0.7861557478368356, 'f1': 0.7429906542056073, 'number': 809} | {'precision': 0.2619047619047619, 'recall': 0.2773109243697479, 'f1': 0.2693877551020408, 'number': 119} | {'precision': 0.7637457044673539, 'recall': 0.8347417840375587, 'f1': 0.7976671152983401, 'number': 1065} | 0.7104 | 0.7817 | 0.7444 | 0.8099 |
| 0.3489 | 10.0 | 100 | 0.6655 | {'precision': 0.6984649122807017, 'recall': 0.7873918417799752, 'f1': 0.7402672864613596, 'number': 809} | {'precision': 0.2714285714285714, 'recall': 0.31932773109243695, 'f1': 0.29343629343629346, 'number': 119} | {'precision': 0.7745614035087719, 'recall': 0.8291079812206573, 'f1': 0.800907029478458, 'number': 1065} | 0.7108 | 0.7817 | 0.7446 | 0.8126 |
| 0.3103 | 11.0 | 110 | 0.6682 | {'precision': 0.6981934112646121, 'recall': 0.8121137206427689, 'f1': 0.7508571428571429, 'number': 809} | {'precision': 0.3173076923076923, 'recall': 0.2773109243697479, 'f1': 0.29596412556053814, 'number': 119} | {'precision': 0.7878521126760564, 'recall': 0.8403755868544601, 'f1': 0.8132666969559291, 'number': 1065} | 0.7267 | 0.7953 | 0.7595 | 0.8148 |
| 0.293 | 12.0 | 120 | 0.6739 | {'precision': 0.7123893805309734, 'recall': 0.796044499381953, 'f1': 0.7518972562755399, 'number': 809} | {'precision': 0.328, 'recall': 0.3445378151260504, 'f1': 0.33606557377049184, 'number': 119} | {'precision': 0.7863475177304965, 'recall': 0.8328638497652582, 'f1': 0.8089375284997721, 'number': 1065} | 0.7288 | 0.7888 | 0.7576 | 0.8167 |
| 0.2761 | 13.0 | 130 | 0.6783 | {'precision': 0.705945945945946, 'recall': 0.8071693448702101, 'f1': 0.7531718569780853, 'number': 809} | {'precision': 0.3467741935483871, 'recall': 0.36134453781512604, 'f1': 0.35390946502057613, 'number': 119} | {'precision': 0.7935656836461126, 'recall': 0.8338028169014085, 'f1': 0.8131868131868133, 'number': 1065} | 0.7306 | 0.7948 | 0.7614 | 0.8137 |
| 0.2633 | 14.0 | 140 | 0.6849 | {'precision': 0.7085590465872156, 'recall': 0.8084054388133498, 'f1': 0.7551963048498845, 'number': 809} | {'precision': 0.31746031746031744, 'recall': 0.33613445378151263, 'f1': 0.32653061224489793, 'number': 119} | {'precision': 0.7883082373782108, 'recall': 0.8356807511737089, 'f1': 0.8113035551504102, 'number': 1065} | 0.7273 | 0.7948 | 0.7595 | 0.8125 |
| 0.2632 | 15.0 | 150 | 0.6847 | {'precision': 0.7144432194046306, 'recall': 0.8009888751545118, 'f1': 0.7552447552447553, 'number': 809} | {'precision': 0.30952380952380953, 'recall': 0.3277310924369748, 'f1': 0.31836734693877555, 'number': 119} | {'precision': 0.7912966252220248, 'recall': 0.8366197183098592, 'f1': 0.81332724783204, 'number': 1065} | 0.7309 | 0.7918 | 0.7601 | 0.8132 |
### Framework versions
- Transformers 4.30.2
- Pytorch 2.0.1+cu118
- Datasets 2.13.1
- Tokenizers 0.13.3
|