File size: 9,319 Bytes
1193c50 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 |
---
license: mit
base_model: microsoft/layoutlm-base-uncased
tags:
- generated_from_trainer
datasets:
- funsd
model-index:
- name: layoutlm-funsd
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# layoutlm-funsd
This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on the funsd dataset.
It achieves the following results on the evaluation set:
- Loss: 1.1459
- Answer: {'precision': 0.3920704845814978, 'recall': 0.5500618046971569, 'f1': 0.45781893004115226, 'number': 809}
- Header: {'precision': 0.36363636363636365, 'recall': 0.2689075630252101, 'f1': 0.30917874396135264, 'number': 119}
- Question: {'precision': 0.5136876006441223, 'recall': 0.5990610328638498, 'f1': 0.553099263112267, 'number': 1065}
- Overall Precision: 0.4523
- Overall Recall: 0.5595
- Overall F1: 0.5002
- Overall Accuracy: 0.6006
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 15
- mixed_precision_training: Native AMP
### Training results
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:-----------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------:|:----------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:|
| 1.7425 | 1.0 | 10 | 1.4798 | {'precision': 0.05438311688311688, 'recall': 0.08281829419035847, 'f1': 0.06565409113179814, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.2244367417677643, 'recall': 0.2431924882629108, 'f1': 0.2334384858044164, 'number': 1065} | 0.1366 | 0.1636 | 0.1489 | 0.3756 |
| 1.419 | 2.0 | 20 | 1.3167 | {'precision': 0.21116377040547657, 'recall': 0.4956736711990111, 'f1': 0.2961595273264402, 'number': 809} | {'precision': 0.08888888888888889, 'recall': 0.03361344537815126, 'f1': 0.048780487804878044, 'number': 119} | {'precision': 0.235467255334805, 'recall': 0.3004694835680751, 'f1': 0.264026402640264, 'number': 1065} | 0.2195 | 0.3638 | 0.2738 | 0.4192 |
| 1.2741 | 3.0 | 30 | 1.2387 | {'precision': 0.2594221105527638, 'recall': 0.5105067985166872, 'f1': 0.34402332361516036, 'number': 809} | {'precision': 0.2702702702702703, 'recall': 0.16806722689075632, 'f1': 0.2072538860103627, 'number': 119} | {'precision': 0.34717494894486045, 'recall': 0.4788732394366197, 'f1': 0.4025256511444357, 'number': 1065} | 0.3008 | 0.4732 | 0.3678 | 0.4611 |
| 1.147 | 4.0 | 40 | 1.1190 | {'precision': 0.26329113924050634, 'recall': 0.5142150803461063, 'f1': 0.34826287149434904, 'number': 809} | {'precision': 0.28, 'recall': 0.17647058823529413, 'f1': 0.21649484536082475, 'number': 119} | {'precision': 0.4030188679245283, 'recall': 0.5014084507042254, 'f1': 0.44686192468619246, 'number': 1065} | 0.3258 | 0.4872 | 0.3905 | 0.5426 |
| 1.0331 | 5.0 | 50 | 1.1534 | {'precision': 0.2893436838390967, 'recall': 0.5067985166872683, 'f1': 0.36837376460017973, 'number': 809} | {'precision': 0.2876712328767123, 'recall': 0.17647058823529413, 'f1': 0.21875000000000003, 'number': 119} | {'precision': 0.4215817694369973, 'recall': 0.5906103286384976, 'f1': 0.4919827923347672, 'number': 1065} | 0.3555 | 0.5319 | 0.4261 | 0.5476 |
| 0.9715 | 6.0 | 60 | 1.1035 | {'precision': 0.3210227272727273, 'recall': 0.5587144622991347, 'f1': 0.4077582318448354, 'number': 809} | {'precision': 0.3157894736842105, 'recall': 0.15126050420168066, 'f1': 0.2045454545454545, 'number': 119} | {'precision': 0.46368243243243246, 'recall': 0.5154929577464789, 'f1': 0.4882169853268119, 'number': 1065} | 0.3847 | 0.5113 | 0.4390 | 0.5706 |
| 0.8925 | 7.0 | 70 | 1.0616 | {'precision': 0.3607266435986159, 'recall': 0.515451174289246, 'f1': 0.42442748091603055, 'number': 809} | {'precision': 0.29473684210526313, 'recall': 0.23529411764705882, 'f1': 0.2616822429906542, 'number': 119} | {'precision': 0.4845360824742268, 'recall': 0.5737089201877934, 'f1': 0.52536543422184, 'number': 1065} | 0.4204 | 0.5299 | 0.4688 | 0.5874 |
| 0.8174 | 8.0 | 80 | 1.0694 | {'precision': 0.3473507148864592, 'recall': 0.5105067985166872, 'f1': 0.4134134134134134, 'number': 809} | {'precision': 0.3373493975903614, 'recall': 0.23529411764705882, 'f1': 0.2772277227722772, 'number': 119} | {'precision': 0.4794414274631497, 'recall': 0.5802816901408451, 'f1': 0.5250637213254036, 'number': 1065} | 0.4135 | 0.5314 | 0.4651 | 0.5893 |
| 0.7698 | 9.0 | 90 | 1.1272 | {'precision': 0.35641227380015733, 'recall': 0.5599505562422744, 'f1': 0.43557692307692303, 'number': 809} | {'precision': 0.3493975903614458, 'recall': 0.24369747899159663, 'f1': 0.2871287128712871, 'number': 119} | {'precision': 0.5008818342151675, 'recall': 0.5333333333333333, 'f1': 0.5165984538426557, 'number': 1065} | 0.4220 | 0.5268 | 0.4686 | 0.5817 |
| 0.7676 | 10.0 | 100 | 1.1380 | {'precision': 0.37153088630259623, 'recall': 0.5129789864029666, 'f1': 0.43094496365524404, 'number': 809} | {'precision': 0.29523809523809524, 'recall': 0.2605042016806723, 'f1': 0.2767857142857143, 'number': 119} | {'precision': 0.5185185185185185, 'recall': 0.5784037558685446, 'f1': 0.546826453617399, 'number': 1065} | 0.4407 | 0.5329 | 0.4824 | 0.5958 |
| 0.6932 | 11.0 | 110 | 1.1051 | {'precision': 0.387, 'recall': 0.4783683559950556, 'f1': 0.42786069651741293, 'number': 809} | {'precision': 0.37037037037037035, 'recall': 0.25210084033613445, 'f1': 0.3, 'number': 119} | {'precision': 0.4865061998541211, 'recall': 0.6262910798122066, 'f1': 0.5476190476190477, 'number': 1065} | 0.4421 | 0.5439 | 0.4877 | 0.6026 |
| 0.6856 | 12.0 | 120 | 1.1257 | {'precision': 0.38833181403828626, 'recall': 0.5265760197775031, 'f1': 0.44700944386149, 'number': 809} | {'precision': 0.3409090909090909, 'recall': 0.25210084033613445, 'f1': 0.2898550724637681, 'number': 119} | {'precision': 0.48674521354933725, 'recall': 0.6206572769953052, 'f1': 0.545604622368964, 'number': 1065} | 0.4392 | 0.5605 | 0.4925 | 0.6021 |
| 0.6592 | 13.0 | 130 | 1.1253 | {'precision': 0.39461883408071746, 'recall': 0.5438813349814586, 'f1': 0.4573804573804573, 'number': 809} | {'precision': 0.3614457831325301, 'recall': 0.25210084033613445, 'f1': 0.297029702970297, 'number': 119} | {'precision': 0.5112179487179487, 'recall': 0.5990610328638498, 'f1': 0.5516645049718979, 'number': 1065} | 0.4530 | 0.5559 | 0.4992 | 0.6066 |
| 0.6358 | 14.0 | 140 | 1.1420 | {'precision': 0.3906810035842294, 'recall': 0.5389369592088998, 'f1': 0.452987012987013, 'number': 809} | {'precision': 0.36904761904761907, 'recall': 0.2605042016806723, 'f1': 0.30541871921182273, 'number': 119} | {'precision': 0.5062597809076682, 'recall': 0.6075117370892019, 'f1': 0.5522833973538199, 'number': 1065} | 0.4496 | 0.5590 | 0.4983 | 0.6018 |
| 0.6263 | 15.0 | 150 | 1.1459 | {'precision': 0.3920704845814978, 'recall': 0.5500618046971569, 'f1': 0.45781893004115226, 'number': 809} | {'precision': 0.36363636363636365, 'recall': 0.2689075630252101, 'f1': 0.30917874396135264, 'number': 119} | {'precision': 0.5136876006441223, 'recall': 0.5990610328638498, 'f1': 0.553099263112267, 'number': 1065} | 0.4523 | 0.5595 | 0.5002 | 0.6006 |
### Framework versions
- Transformers 4.39.3
- Pytorch 2.2.2+cu121
- Datasets 2.18.0
- Tokenizers 0.15.2
|