File size: 9,366 Bytes
db3a82f | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 | ---
license: mit
base_model: microsoft/layoutlm-base-uncased
tags:
- generated_from_trainer
datasets:
- funsd
model-index:
- name: layoutlm-funsd
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# layoutlm-funsd
This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on the funsd dataset.
It achieves the following results on the evaluation set:
- Loss: 0.7099
- Answer: {'precision': 0.7126948775055679, 'recall': 0.7911001236093943, 'f1': 0.7498535442296427, 'number': 809}
- Header: {'precision': 0.3793103448275862, 'recall': 0.3697478991596639, 'f1': 0.374468085106383, 'number': 119}
- Question: {'precision': 0.7863397548161121, 'recall': 0.8431924882629108, 'f1': 0.813774354327141, 'number': 1065}
- Overall Precision: 0.7338
- Overall Recall: 0.7938
- Overall F1: 0.7626
- Overall Accuracy: 0.8008
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 15
- mixed_precision_training: Native AMP
### Training results
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:--------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:|
| 1.8413 | 1.0 | 10 | 1.6504 | {'precision': 0.011467889908256881, 'recall': 0.006180469715698393, 'f1': 0.008032128514056226, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.2831168831168831, 'recall': 0.10234741784037558, 'f1': 0.15034482758620688, 'number': 1065} | 0.1389 | 0.0572 | 0.0810 | 0.3247 |
| 1.5029 | 2.0 | 20 | 1.3220 | {'precision': 0.1353811149032992, 'recall': 0.14709517923362175, 'f1': 0.1409952606635071, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.40875370919881304, 'recall': 0.5173708920187794, 'f1': 0.4566929133858268, 'number': 1065} | 0.3009 | 0.3362 | 0.3175 | 0.5584 |
| 1.1608 | 3.0 | 30 | 1.0033 | {'precision': 0.4221267454350161, 'recall': 0.4857849196538937, 'f1': 0.45172413793103444, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.5333333333333333, 'recall': 0.6910798122065728, 'f1': 0.6020449897750512, 'number': 1065} | 0.4871 | 0.5665 | 0.5238 | 0.6833 |
| 0.9008 | 4.0 | 40 | 0.8415 | {'precision': 0.5449101796407185, 'recall': 0.6749072929542645, 'f1': 0.6029817780231916, 'number': 809} | {'precision': 0.11904761904761904, 'recall': 0.04201680672268908, 'f1': 0.06211180124223603, 'number': 119} | {'precision': 0.6373355263157895, 'recall': 0.7276995305164319, 'f1': 0.6795265234546252, 'number': 1065} | 0.5867 | 0.6653 | 0.6236 | 0.7396 |
| 0.7085 | 5.0 | 50 | 0.7531 | {'precision': 0.6281628162816282, 'recall': 0.7058096415327565, 'f1': 0.6647264260768335, 'number': 809} | {'precision': 0.16, 'recall': 0.10084033613445378, 'f1': 0.12371134020618556, 'number': 119} | {'precision': 0.6728110599078341, 'recall': 0.8225352112676056, 'f1': 0.7401774397972117, 'number': 1065} | 0.6382 | 0.7321 | 0.6819 | 0.7708 |
| 0.6016 | 6.0 | 60 | 0.7178 | {'precision': 0.6607515657620042, 'recall': 0.7824474660074165, 'f1': 0.7164685908319186, 'number': 809} | {'precision': 0.2235294117647059, 'recall': 0.15966386554621848, 'f1': 0.18627450980392157, 'number': 119} | {'precision': 0.7422145328719724, 'recall': 0.8056338028169014, 'f1': 0.7726249437190456, 'number': 1065} | 0.6867 | 0.7577 | 0.7204 | 0.7819 |
| 0.5255 | 7.0 | 70 | 0.6773 | {'precision': 0.6886688668866887, 'recall': 0.7737948084054388, 'f1': 0.7287543655413271, 'number': 809} | {'precision': 0.3235294117647059, 'recall': 0.2773109243697479, 'f1': 0.2986425339366516, 'number': 119} | {'precision': 0.748932536293766, 'recall': 0.8234741784037559, 'f1': 0.7844364937388193, 'number': 1065} | 0.7039 | 0.7707 | 0.7358 | 0.7985 |
| 0.4664 | 8.0 | 80 | 0.6865 | {'precision': 0.6846652267818575, 'recall': 0.7836835599505563, 'f1': 0.730835734870317, 'number': 809} | {'precision': 0.24299065420560748, 'recall': 0.2184873949579832, 'f1': 0.2300884955752212, 'number': 119} | {'precision': 0.7593397046046916, 'recall': 0.8206572769953052, 'f1': 0.7888086642599278, 'number': 1065} | 0.7024 | 0.7697 | 0.7345 | 0.7950 |
| 0.4092 | 9.0 | 90 | 0.6843 | {'precision': 0.6929046563192904, 'recall': 0.7725587144622992, 'f1': 0.7305669199298657, 'number': 809} | {'precision': 0.3050847457627119, 'recall': 0.3025210084033613, 'f1': 0.3037974683544304, 'number': 119} | {'precision': 0.7587085811384877, 'recall': 0.8384976525821596, 'f1': 0.7966101694915255, 'number': 1065} | 0.7073 | 0.7797 | 0.7418 | 0.8013 |
| 0.4007 | 10.0 | 100 | 0.6826 | {'precision': 0.6887921653971708, 'recall': 0.7824474660074165, 'f1': 0.7326388888888888, 'number': 809} | {'precision': 0.3142857142857143, 'recall': 0.2773109243697479, 'f1': 0.29464285714285715, 'number': 119} | {'precision': 0.7761578044596913, 'recall': 0.8497652582159625, 'f1': 0.811295383236217, 'number': 1065} | 0.7174 | 0.7883 | 0.7511 | 0.8001 |
| 0.3396 | 11.0 | 110 | 0.6904 | {'precision': 0.6922246220302376, 'recall': 0.792336217552534, 'f1': 0.7389048991354467, 'number': 809} | {'precision': 0.32231404958677684, 'recall': 0.3277310924369748, 'f1': 0.32499999999999996, 'number': 119} | {'precision': 0.7778745644599303, 'recall': 0.8384976525821596, 'f1': 0.8070492544057841, 'number': 1065} | 0.7166 | 0.7893 | 0.7512 | 0.8036 |
| 0.3223 | 12.0 | 120 | 0.7032 | {'precision': 0.7138084632516704, 'recall': 0.792336217552534, 'f1': 0.7510251903925014, 'number': 809} | {'precision': 0.3669724770642202, 'recall': 0.33613445378151263, 'f1': 0.3508771929824562, 'number': 119} | {'precision': 0.788546255506608, 'recall': 0.8403755868544601, 'f1': 0.8136363636363636, 'number': 1065} | 0.7358 | 0.7908 | 0.7623 | 0.8012 |
| 0.3079 | 13.0 | 130 | 0.7098 | {'precision': 0.6950431034482759, 'recall': 0.7972805933250927, 'f1': 0.7426597582037997, 'number': 809} | {'precision': 0.3652173913043478, 'recall': 0.35294117647058826, 'f1': 0.35897435897435903, 'number': 119} | {'precision': 0.7906360424028268, 'recall': 0.8403755868544601, 'f1': 0.81474738279472, 'number': 1065} | 0.7274 | 0.7938 | 0.7591 | 0.8027 |
| 0.2866 | 14.0 | 140 | 0.7096 | {'precision': 0.7103218645948945, 'recall': 0.7911001236093943, 'f1': 0.7485380116959064, 'number': 809} | {'precision': 0.36065573770491804, 'recall': 0.3697478991596639, 'f1': 0.36514522821576767, 'number': 119} | {'precision': 0.787719298245614, 'recall': 0.8431924882629108, 'f1': 0.8145124716553288, 'number': 1065} | 0.7314 | 0.7938 | 0.7613 | 0.8007 |
| 0.2847 | 15.0 | 150 | 0.7099 | {'precision': 0.7126948775055679, 'recall': 0.7911001236093943, 'f1': 0.7498535442296427, 'number': 809} | {'precision': 0.3793103448275862, 'recall': 0.3697478991596639, 'f1': 0.374468085106383, 'number': 119} | {'precision': 0.7863397548161121, 'recall': 0.8431924882629108, 'f1': 0.813774354327141, 'number': 1065} | 0.7338 | 0.7938 | 0.7626 | 0.8008 |
### Framework versions
- Transformers 4.41.2
- Pytorch 2.3.0+cu121
- Datasets 2.20.0
- Tokenizers 0.19.1
|