File size: 16,321 Bytes
09b2e52 690c316 09b2e52 690c316 09b2e52 690c316 09b2e52 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 | ---
library_name: transformers
license: mit
base_model: microsoft/layoutlm-base-uncased
tags:
- generated_from_trainer
datasets:
- funsd
model-index:
- name: layoutlm-funsd
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# layoutlm-funsd
This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on the funsd dataset.
It achieves the following results on the evaluation set:
- Loss: 0.9059
- Answer: {'precision': 0.7288693743139407, 'recall': 0.8207663782447466, 'f1': 0.772093023255814, 'number': 809}
- Header: {'precision': 0.43795620437956206, 'recall': 0.5042016806722689, 'f1': 0.46875, 'number': 119}
- Question: {'precision': 0.8137614678899082, 'recall': 0.8328638497652582, 'f1': 0.8232018561484918, 'number': 1065}
- Overall Precision: 0.7535
- Overall Recall: 0.8083
- Overall F1: 0.7800
- Overall Accuracy: 0.8069
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- num_epochs: 30
- mixed_precision_training: Native AMP
### Training results
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:-------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:|
| 1.8292 | 1.0 | 10 | 1.5818 | {'precision': 0.00816326530612245, 'recall': 0.007416563658838072, 'f1': 0.007772020725388602, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.22530864197530864, 'recall': 0.13708920187793427, 'f1': 0.17046117921774664, 'number': 1065} | 0.1099 | 0.0763 | 0.0900 | 0.3514 |
| 1.4621 | 2.0 | 20 | 1.2540 | {'precision': 0.2941659819227609, 'recall': 0.44252163164400493, 'f1': 0.3534057255676209, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.391804457225018, 'recall': 0.5117370892018779, 'f1': 0.44381107491856675, 'number': 1065} | 0.3451 | 0.4531 | 0.3918 | 0.6061 |
| 1.0998 | 3.0 | 30 | 0.9167 | {'precision': 0.5, 'recall': 0.6254635352286774, 'f1': 0.5557386051619989, 'number': 809} | {'precision': 0.058823529411764705, 'recall': 0.025210084033613446, 'f1': 0.03529411764705882, 'number': 119} | {'precision': 0.5253118121790169, 'recall': 0.672300469483568, 'f1': 0.5897858319604613, 'number': 1065} | 0.5049 | 0.6147 | 0.5544 | 0.7112 |
| 0.8275 | 4.0 | 40 | 0.7689 | {'precision': 0.5742667928098392, 'recall': 0.7503090234857849, 'f1': 0.6505894962486602, 'number': 809} | {'precision': 0.2894736842105263, 'recall': 0.18487394957983194, 'f1': 0.22564102564102564, 'number': 119} | {'precision': 0.6544315129811996, 'recall': 0.6863849765258216, 'f1': 0.6700274977085243, 'number': 1065} | 0.6044 | 0.6824 | 0.6411 | 0.7610 |
| 0.6622 | 5.0 | 50 | 0.7165 | {'precision': 0.6455424274973147, 'recall': 0.7428924598269468, 'f1': 0.6908045977011494, 'number': 809} | {'precision': 0.36363636363636365, 'recall': 0.23529411764705882, 'f1': 0.2857142857142857, 'number': 119} | {'precision': 0.6690647482014388, 'recall': 0.7859154929577464, 'f1': 0.7227979274611398, 'number': 1065} | 0.6490 | 0.7356 | 0.6896 | 0.7792 |
| 0.5529 | 6.0 | 60 | 0.6745 | {'precision': 0.6523955147808359, 'recall': 0.7911001236093943, 'f1': 0.7150837988826816, 'number': 809} | {'precision': 0.3473684210526316, 'recall': 0.2773109243697479, 'f1': 0.308411214953271, 'number': 119} | {'precision': 0.726649528706084, 'recall': 0.7962441314553991, 'f1': 0.7598566308243728, 'number': 1065} | 0.6781 | 0.7632 | 0.7181 | 0.7876 |
| 0.4607 | 7.0 | 70 | 0.6568 | {'precision': 0.6858359957401491, 'recall': 0.796044499381953, 'f1': 0.7368421052631579, 'number': 809} | {'precision': 0.3490566037735849, 'recall': 0.31092436974789917, 'f1': 0.32888888888888884, 'number': 119} | {'precision': 0.7552447552447552, 'recall': 0.8112676056338028, 'f1': 0.7822544137618832, 'number': 1065} | 0.7058 | 0.7752 | 0.7389 | 0.7954 |
| 0.398 | 8.0 | 80 | 0.6752 | {'precision': 0.6826722338204593, 'recall': 0.8084054388133498, 'f1': 0.7402376910016978, 'number': 809} | {'precision': 0.3333333333333333, 'recall': 0.31932773109243695, 'f1': 0.3261802575107296, 'number': 119} | {'precision': 0.7607361963190185, 'recall': 0.8150234741784037, 'f1': 0.786944696282865, 'number': 1065} | 0.7049 | 0.7827 | 0.7418 | 0.7957 |
| 0.3405 | 9.0 | 90 | 0.6934 | {'precision': 0.6829015544041451, 'recall': 0.8145859085290482, 'f1': 0.7429537767756483, 'number': 809} | {'precision': 0.3412698412698413, 'recall': 0.36134453781512604, 'f1': 0.35102040816326535, 'number': 119} | {'precision': 0.7694369973190348, 'recall': 0.8084507042253521, 'f1': 0.7884615384615383, 'number': 1065} | 0.7072 | 0.7842 | 0.7438 | 0.7965 |
| 0.3225 | 10.0 | 100 | 0.7087 | {'precision': 0.6884899683210137, 'recall': 0.8059332509270705, 'f1': 0.7425968109339408, 'number': 809} | {'precision': 0.35714285714285715, 'recall': 0.33613445378151263, 'f1': 0.34632034632034636, 'number': 119} | {'precision': 0.7785714285714286, 'recall': 0.8187793427230047, 'f1': 0.7981693363844393, 'number': 1065} | 0.7178 | 0.7847 | 0.7498 | 0.7999 |
| 0.2555 | 11.0 | 110 | 0.7154 | {'precision': 0.7044967880085653, 'recall': 0.8133498145859085, 'f1': 0.7550200803212851, 'number': 809} | {'precision': 0.40384615384615385, 'recall': 0.35294117647058826, 'f1': 0.3766816143497759, 'number': 119} | {'precision': 0.7887197851387645, 'recall': 0.8272300469483568, 'f1': 0.8075160403299725, 'number': 1065} | 0.7336 | 0.7933 | 0.7623 | 0.8047 |
| 0.2238 | 12.0 | 120 | 0.7295 | {'precision': 0.7291666666666666, 'recall': 0.8220024721878862, 'f1': 0.7728065078442764, 'number': 809} | {'precision': 0.3793103448275862, 'recall': 0.3697478991596639, 'f1': 0.374468085106383, 'number': 119} | {'precision': 0.7824194952132288, 'recall': 0.844131455399061, 'f1': 0.8121047877145439, 'number': 1065} | 0.7386 | 0.8068 | 0.7712 | 0.8082 |
| 0.198 | 13.0 | 130 | 0.7615 | {'precision': 0.7092651757188498, 'recall': 0.823238566131026, 'f1': 0.7620137299771167, 'number': 809} | {'precision': 0.41346153846153844, 'recall': 0.36134453781512604, 'f1': 0.3856502242152467, 'number': 119} | {'precision': 0.8021680216802168, 'recall': 0.8338028169014085, 'f1': 0.8176795580110497, 'number': 1065} | 0.7428 | 0.8013 | 0.7709 | 0.8060 |
| 0.1691 | 14.0 | 140 | 0.7624 | {'precision': 0.7232432432432433, 'recall': 0.826946847960445, 'f1': 0.7716262975778547, 'number': 809} | {'precision': 0.4090909090909091, 'recall': 0.37815126050420167, 'f1': 0.39301310043668125, 'number': 119} | {'precision': 0.7978436657681941, 'recall': 0.8338028169014085, 'f1': 0.8154269972451792, 'number': 1065} | 0.7458 | 0.8038 | 0.7737 | 0.8108 |
| 0.1504 | 15.0 | 150 | 0.7685 | {'precision': 0.7197802197802198, 'recall': 0.8096415327564895, 'f1': 0.7620709714950552, 'number': 809} | {'precision': 0.41228070175438597, 'recall': 0.3949579831932773, 'f1': 0.40343347639484983, 'number': 119} | {'precision': 0.8028802880288028, 'recall': 0.8375586854460094, 'f1': 0.8198529411764706, 'number': 1065} | 0.7466 | 0.7998 | 0.7723 | 0.8104 |
| 0.1387 | 16.0 | 160 | 0.8119 | {'precision': 0.7063740856844305, 'recall': 0.8355995055624228, 'f1': 0.7655719139297849, 'number': 809} | {'precision': 0.38405797101449274, 'recall': 0.44537815126050423, 'f1': 0.41245136186770426, 'number': 119} | {'precision': 0.7994530537830447, 'recall': 0.8234741784037559, 'f1': 0.8112858464384829, 'number': 1065} | 0.7327 | 0.8058 | 0.7675 | 0.8045 |
| 0.1289 | 17.0 | 170 | 0.8040 | {'precision': 0.7145922746781116, 'recall': 0.823238566131026, 'f1': 0.7650775416427341, 'number': 809} | {'precision': 0.4236111111111111, 'recall': 0.5126050420168067, 'f1': 0.4638783269961977, 'number': 119} | {'precision': 0.819718309859155, 'recall': 0.819718309859155, 'f1': 0.819718309859155, 'number': 1065} | 0.7473 | 0.8028 | 0.7741 | 0.8078 |
| 0.1113 | 18.0 | 180 | 0.8194 | {'precision': 0.732662192393736, 'recall': 0.8096415327564895, 'f1': 0.7692307692307693, 'number': 809} | {'precision': 0.4233576642335766, 'recall': 0.48739495798319327, 'f1': 0.453125, 'number': 119} | {'precision': 0.8119891008174387, 'recall': 0.8394366197183099, 'f1': 0.8254847645429363, 'number': 1065} | 0.7538 | 0.8063 | 0.7792 | 0.8082 |
| 0.1034 | 19.0 | 190 | 0.8405 | {'precision': 0.721205597416577, 'recall': 0.8281829419035847, 'f1': 0.7710011507479863, 'number': 809} | {'precision': 0.4153846153846154, 'recall': 0.453781512605042, 'f1': 0.43373493975903615, 'number': 119} | {'precision': 0.8083941605839416, 'recall': 0.831924882629108, 'f1': 0.8199907450254512, 'number': 1065} | 0.7471 | 0.8078 | 0.7763 | 0.8076 |
| 0.0948 | 20.0 | 200 | 0.8530 | {'precision': 0.7199124726477024, 'recall': 0.8133498145859085, 'f1': 0.7637840975043529, 'number': 809} | {'precision': 0.4117647058823529, 'recall': 0.47058823529411764, 'f1': 0.4392156862745098, 'number': 119} | {'precision': 0.8070333633904418, 'recall': 0.8403755868544601, 'f1': 0.8233670653173873, 'number': 1065} | 0.7453 | 0.8073 | 0.7750 | 0.8092 |
| 0.0868 | 21.0 | 210 | 0.8617 | {'precision': 0.7291666666666666, 'recall': 0.8220024721878862, 'f1': 0.7728065078442764, 'number': 809} | {'precision': 0.4198473282442748, 'recall': 0.46218487394957986, 'f1': 0.43999999999999995, 'number': 119} | {'precision': 0.8148487626031164, 'recall': 0.8347417840375587, 'f1': 0.8246753246753246, 'number': 1065} | 0.7540 | 0.8073 | 0.7797 | 0.8083 |
| 0.0905 | 22.0 | 220 | 0.8748 | {'precision': 0.7333333333333333, 'recall': 0.8158220024721878, 'f1': 0.7723815096547689, 'number': 809} | {'precision': 0.39215686274509803, 'recall': 0.5042016806722689, 'f1': 0.4411764705882353, 'number': 119} | {'precision': 0.813126709206928, 'recall': 0.8375586854460094, 'f1': 0.8251618871415356, 'number': 1065} | 0.7498 | 0.8088 | 0.7782 | 0.8064 |
| 0.0809 | 23.0 | 230 | 0.8749 | {'precision': 0.724025974025974, 'recall': 0.826946847960445, 'f1': 0.7720715522215811, 'number': 809} | {'precision': 0.44274809160305345, 'recall': 0.48739495798319327, 'f1': 0.464, 'number': 119} | {'precision': 0.8132474701011959, 'recall': 0.8300469483568075, 'f1': 0.8215613382899627, 'number': 1065} | 0.7521 | 0.8083 | 0.7792 | 0.8087 |
| 0.073 | 24.0 | 240 | 0.8752 | {'precision': 0.7290748898678414, 'recall': 0.8182941903584673, 'f1': 0.7711124053581829, 'number': 809} | {'precision': 0.43703703703703706, 'recall': 0.4957983193277311, 'f1': 0.4645669291338583, 'number': 119} | {'precision': 0.8156934306569343, 'recall': 0.8394366197183099, 'f1': 0.8273947246645073, 'number': 1065} | 0.7550 | 0.8103 | 0.7817 | 0.8090 |
| 0.0694 | 25.0 | 250 | 0.8898 | {'precision': 0.723986856516977, 'recall': 0.8170580964153276, 'f1': 0.7677119628339142, 'number': 809} | {'precision': 0.427536231884058, 'recall': 0.4957983193277311, 'f1': 0.4591439688715953, 'number': 119} | {'precision': 0.8108356290174472, 'recall': 0.8291079812206573, 'f1': 0.819870009285051, 'number': 1065} | 0.7491 | 0.8043 | 0.7757 | 0.8070 |
| 0.0726 | 26.0 | 260 | 0.8944 | {'precision': 0.7213656387665198, 'recall': 0.8096415327564895, 'f1': 0.762958648806057, 'number': 809} | {'precision': 0.43703703703703706, 'recall': 0.4957983193277311, 'f1': 0.4645669291338583, 'number': 119} | {'precision': 0.8170173833485819, 'recall': 0.8384976525821596, 'f1': 0.8276181649675627, 'number': 1065} | 0.7523 | 0.8063 | 0.7784 | 0.8082 |
| 0.0674 | 27.0 | 270 | 0.9073 | {'precision': 0.7337733773377337, 'recall': 0.8244746600741656, 'f1': 0.7764842840512223, 'number': 809} | {'precision': 0.41304347826086957, 'recall': 0.4789915966386555, 'f1': 0.443579766536965, 'number': 119} | {'precision': 0.8118721461187215, 'recall': 0.8347417840375587, 'f1': 0.8231481481481482, 'number': 1065} | 0.7530 | 0.8093 | 0.7802 | 0.8057 |
| 0.0701 | 28.0 | 280 | 0.9131 | {'precision': 0.7257889009793254, 'recall': 0.8244746600741656, 'f1': 0.7719907407407407, 'number': 809} | {'precision': 0.4195804195804196, 'recall': 0.5042016806722689, 'f1': 0.4580152671755725, 'number': 119} | {'precision': 0.8141674333026679, 'recall': 0.8309859154929577, 'f1': 0.8224907063197026, 'number': 1065} | 0.7501 | 0.8088 | 0.7784 | 0.8055 |
| 0.0657 | 29.0 | 290 | 0.9060 | {'precision': 0.7304730473047305, 'recall': 0.8207663782447466, 'f1': 0.7729918509895226, 'number': 809} | {'precision': 0.44029850746268656, 'recall': 0.4957983193277311, 'f1': 0.46640316205533594, 'number': 119} | {'precision': 0.8119266055045872, 'recall': 0.8309859154929577, 'f1': 0.8213457076566124, 'number': 1065} | 0.7539 | 0.8068 | 0.7794 | 0.8075 |
| 0.0641 | 30.0 | 300 | 0.9059 | {'precision': 0.7288693743139407, 'recall': 0.8207663782447466, 'f1': 0.772093023255814, 'number': 809} | {'precision': 0.43795620437956206, 'recall': 0.5042016806722689, 'f1': 0.46875, 'number': 119} | {'precision': 0.8137614678899082, 'recall': 0.8328638497652582, 'f1': 0.8232018561484918, 'number': 1065} | 0.7535 | 0.8083 | 0.7800 | 0.8069 |
### Framework versions
- Transformers 4.47.1
- Pytorch 2.5.1+cu124
- Datasets 3.2.0
- Tokenizers 0.21.0
|