File size: 9,478 Bytes
6bceabf | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 | ---
library_name: transformers
license: mit
base_model: microsoft/layoutlm-base-uncased
tags:
- generated_from_trainer
datasets:
- funsd
model-index:
- name: layoutlm-funsd
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# layoutlm-funsd
This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on the funsd dataset.
It achieves the following results on the evaluation set:
- Loss: 0.6975
- Answer: {'precision': 0.7057522123893806, 'recall': 0.788627935723115, 'f1': 0.7448920023350847, 'number': 809}
- Header: {'precision': 0.2748091603053435, 'recall': 0.3025210084033613, 'f1': 0.288, 'number': 119}
- Question: {'precision': 0.7804232804232805, 'recall': 0.8309859154929577, 'f1': 0.8049113233287858, 'number': 1065}
- Overall Precision: 0.7188
- Overall Recall: 0.7822
- Overall F1: 0.7492
- Overall Accuracy: 0.8031
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- num_epochs: 15
- mixed_precision_training: Native AMP
### Training results
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:--------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:|
| 1.8422 | 1.0 | 10 | 1.6206 | {'precision': 0.025280898876404494, 'recall': 0.022249690976514216, 'f1': 0.023668639053254437, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.2301255230125523, 'recall': 0.15492957746478872, 'f1': 0.18518518518518517, 'number': 1065} | 0.1277 | 0.0918 | 0.1068 | 0.3520 |
| 1.4509 | 2.0 | 20 | 1.2639 | {'precision': 0.15125, 'recall': 0.14956736711990112, 'f1': 0.15040397762585456, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.465642683912692, 'recall': 0.5408450704225352, 'f1': 0.5004344048653345, 'number': 1065} | 0.3420 | 0.3497 | 0.3458 | 0.5726 |
| 1.1141 | 3.0 | 30 | 0.9554 | {'precision': 0.5079872204472844, 'recall': 0.5896168108776267, 'f1': 0.5457665903890161, 'number': 809} | {'precision': 0.07894736842105263, 'recall': 0.025210084033613446, 'f1': 0.038216560509554146, 'number': 119} | {'precision': 0.6070007955449482, 'recall': 0.7164319248826291, 'f1': 0.6571920757967269, 'number': 1065} | 0.5564 | 0.6237 | 0.5881 | 0.7246 |
| 0.8541 | 4.0 | 40 | 0.7774 | {'precision': 0.5957446808510638, 'recall': 0.69221260815822, 'f1': 0.6403659233847913, 'number': 809} | {'precision': 0.19117647058823528, 'recall': 0.1092436974789916, 'f1': 0.13903743315508021, 'number': 119} | {'precision': 0.6491228070175439, 'recall': 0.7643192488262911, 'f1': 0.7020267356619233, 'number': 1065} | 0.6132 | 0.6959 | 0.6519 | 0.7636 |
| 0.6793 | 5.0 | 50 | 0.7244 | {'precision': 0.6372549019607843, 'recall': 0.723114956736712, 'f1': 0.677475390851187, 'number': 809} | {'precision': 0.23076923076923078, 'recall': 0.15126050420168066, 'f1': 0.18274111675126906, 'number': 119} | {'precision': 0.6705516705516705, 'recall': 0.8103286384976526, 'f1': 0.733843537414966, 'number': 1065} | 0.6421 | 0.7356 | 0.6857 | 0.7706 |
| 0.5888 | 6.0 | 60 | 0.6842 | {'precision': 0.6595517609391676, 'recall': 0.7639060568603214, 'f1': 0.7079037800687286, 'number': 809} | {'precision': 0.2597402597402597, 'recall': 0.16806722689075632, 'f1': 0.20408163265306123, 'number': 119} | {'precision': 0.7196339434276207, 'recall': 0.812206572769953, 'f1': 0.7631230701367445, 'number': 1065} | 0.6782 | 0.7541 | 0.7142 | 0.7847 |
| 0.5107 | 7.0 | 70 | 0.6648 | {'precision': 0.6694473409801877, 'recall': 0.7935723114956736, 'f1': 0.7262443438914026, 'number': 809} | {'precision': 0.24561403508771928, 'recall': 0.23529411764705882, 'f1': 0.24034334763948498, 'number': 119} | {'precision': 0.7504347826086957, 'recall': 0.8103286384976526, 'f1': 0.779232505643341, 'number': 1065} | 0.6896 | 0.7692 | 0.7272 | 0.7944 |
| 0.455 | 8.0 | 80 | 0.6582 | {'precision': 0.6750788643533123, 'recall': 0.7935723114956736, 'f1': 0.7295454545454545, 'number': 809} | {'precision': 0.25961538461538464, 'recall': 0.226890756302521, 'f1': 0.242152466367713, 'number': 119} | {'precision': 0.75, 'recall': 0.8253521126760563, 'f1': 0.7858739383102369, 'number': 1065} | 0.6951 | 0.7767 | 0.7336 | 0.7990 |
| 0.4012 | 9.0 | 90 | 0.6598 | {'precision': 0.6955093099671413, 'recall': 0.7849196538936959, 'f1': 0.7375145180023228, 'number': 809} | {'precision': 0.25203252032520324, 'recall': 0.2605042016806723, 'f1': 0.25619834710743805, 'number': 119} | {'precision': 0.7566409597257926, 'recall': 0.8291079812206573, 'f1': 0.7912186379928315, 'number': 1065} | 0.7031 | 0.7772 | 0.7383 | 0.7989 |
| 0.3934 | 10.0 | 100 | 0.6706 | {'precision': 0.7069716775599129, 'recall': 0.8022249690976514, 'f1': 0.751592356687898, 'number': 809} | {'precision': 0.27586206896551724, 'recall': 0.2689075630252101, 'f1': 0.27234042553191484, 'number': 119} | {'precision': 0.7775784753363228, 'recall': 0.8140845070422535, 'f1': 0.7954128440366972, 'number': 1065} | 0.7203 | 0.7767 | 0.7475 | 0.8052 |
| 0.3354 | 11.0 | 110 | 0.6780 | {'precision': 0.7046688382193268, 'recall': 0.8022249690976514, 'f1': 0.7502890173410405, 'number': 809} | {'precision': 0.272, 'recall': 0.2857142857142857, 'f1': 0.27868852459016397, 'number': 119} | {'precision': 0.7675628794449263, 'recall': 0.8309859154929577, 'f1': 0.7980162308385933, 'number': 1065} | 0.7131 | 0.7868 | 0.7481 | 0.8028 |
| 0.3187 | 12.0 | 120 | 0.6842 | {'precision': 0.7097130242825607, 'recall': 0.7948084054388134, 'f1': 0.7498542274052479, 'number': 809} | {'precision': 0.26717557251908397, 'recall': 0.29411764705882354, 'f1': 0.28, 'number': 119} | {'precision': 0.7742504409171076, 'recall': 0.8244131455399061, 'f1': 0.7985447930877672, 'number': 1065} | 0.7167 | 0.7807 | 0.7474 | 0.8021 |
| 0.3007 | 13.0 | 130 | 0.6944 | {'precision': 0.704225352112676, 'recall': 0.8034610630407911, 'f1': 0.7505773672055426, 'number': 809} | {'precision': 0.2833333333333333, 'recall': 0.2857142857142857, 'f1': 0.2845188284518828, 'number': 119} | {'precision': 0.7791519434628975, 'recall': 0.828169014084507, 'f1': 0.8029130632680928, 'number': 1065} | 0.72 | 0.7858 | 0.7514 | 0.8022 |
| 0.2805 | 14.0 | 140 | 0.7007 | {'precision': 0.7126948775055679, 'recall': 0.7911001236093943, 'f1': 0.7498535442296427, 'number': 809} | {'precision': 0.2695035460992908, 'recall': 0.31932773109243695, 'f1': 0.2923076923076923, 'number': 119} | {'precision': 0.7805530776092774, 'recall': 0.8215962441314554, 'f1': 0.8005489478499542, 'number': 1065} | 0.7190 | 0.7792 | 0.7479 | 0.8010 |
| 0.2795 | 15.0 | 150 | 0.6975 | {'precision': 0.7057522123893806, 'recall': 0.788627935723115, 'f1': 0.7448920023350847, 'number': 809} | {'precision': 0.2748091603053435, 'recall': 0.3025210084033613, 'f1': 0.288, 'number': 119} | {'precision': 0.7804232804232805, 'recall': 0.8309859154929577, 'f1': 0.8049113233287858, 'number': 1065} | 0.7188 | 0.7822 | 0.7492 | 0.8031 |
### Framework versions
- Transformers 4.46.3
- Pytorch 2.5.1+cu121
- Datasets 3.2.0
- Tokenizers 0.20.3
|