File size: 9,350 Bytes
cd6b4c7 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 | ---
license: mit
base_model: microsoft/layoutlm-base-uncased
tags:
- generated_from_trainer
datasets:
- funsd
model-index:
- name: layoutlm-funsd
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# layoutlm-funsd
This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on the funsd dataset.
It achieves the following results on the evaluation set:
- Loss: 0.7045
- Answer: {'precision': 0.7013274336283186, 'recall': 0.7836835599505563, 'f1': 0.7402218330414477, 'number': 809}
- Header: {'precision': 0.3111111111111111, 'recall': 0.35294117647058826, 'f1': 0.33070866141732286, 'number': 119}
- Question: {'precision': 0.773936170212766, 'recall': 0.819718309859155, 'f1': 0.796169630642955, 'number': 1065}
- Overall Precision: 0.7148
- Overall Recall: 0.7772
- Overall F1: 0.7447
- Overall Accuracy: 0.8045
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 15
- mixed_precision_training: Native AMP
### Training results
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:-----------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:|
| 1.8311 | 1.0 | 10 | 1.5893 | {'precision': 0.01643192488262911, 'recall': 0.0173053152039555, 'f1': 0.016857314870559904, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.23476005188067445, 'recall': 0.1699530516431925, 'f1': 0.19716775599128541, 'number': 1065} | 0.1201 | 0.0978 | 0.1079 | 0.3735 |
| 1.453 | 2.0 | 20 | 1.2320 | {'precision': 0.16265750286368844, 'recall': 0.17552533992583436, 'f1': 0.16884661117717004, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.43330980945659847, 'recall': 0.5765258215962441, 'f1': 0.49476228847703463, 'number': 1065} | 0.3301 | 0.3793 | 0.3530 | 0.5937 |
| 1.0885 | 3.0 | 30 | 0.9242 | {'precision': 0.4983991462113127, 'recall': 0.5772558714462299, 'f1': 0.5349369988545246, 'number': 809} | {'precision': 0.05263157894736842, 'recall': 0.008403361344537815, 'f1': 0.014492753623188406, 'number': 119} | {'precision': 0.556745182012848, 'recall': 0.7323943661971831, 'f1': 0.632603406326034, 'number': 1065} | 0.5295 | 0.6262 | 0.5738 | 0.7132 |
| 0.8351 | 4.0 | 40 | 0.7984 | {'precision': 0.5991902834008097, 'recall': 0.7317676143386898, 'f1': 0.6588759042849194, 'number': 809} | {'precision': 0.16, 'recall': 0.06722689075630252, 'f1': 0.09467455621301775, 'number': 119} | {'precision': 0.6562763268744735, 'recall': 0.7314553990610329, 'f1': 0.691829484902309, 'number': 1065} | 0.6198 | 0.6919 | 0.6539 | 0.7574 |
| 0.6746 | 5.0 | 50 | 0.7364 | {'precision': 0.6524663677130045, 'recall': 0.7194066749072929, 'f1': 0.6843033509700176, 'number': 809} | {'precision': 0.21951219512195122, 'recall': 0.15126050420168066, 'f1': 0.1791044776119403, 'number': 119} | {'precision': 0.6493212669683258, 'recall': 0.8084507042253521, 'f1': 0.7202007528230866, 'number': 1065} | 0.6352 | 0.7331 | 0.6806 | 0.7789 |
| 0.5833 | 6.0 | 60 | 0.7065 | {'precision': 0.6387487386478304, 'recall': 0.7824474660074165, 'f1': 0.7033333333333333, 'number': 809} | {'precision': 0.25333333333333335, 'recall': 0.15966386554621848, 'f1': 0.1958762886597938, 'number': 119} | {'precision': 0.7177489177489178, 'recall': 0.7784037558685446, 'f1': 0.7468468468468469, 'number': 1065} | 0.6668 | 0.7431 | 0.7029 | 0.7837 |
| 0.5101 | 7.0 | 70 | 0.6765 | {'precision': 0.6811751904243744, 'recall': 0.7737948084054388, 'f1': 0.724537037037037, 'number': 809} | {'precision': 0.2564102564102564, 'recall': 0.25210084033613445, 'f1': 0.2542372881355932, 'number': 119} | {'precision': 0.7319762510602206, 'recall': 0.8103286384976526, 'f1': 0.7691622103386809, 'number': 1065} | 0.6858 | 0.7622 | 0.7220 | 0.7972 |
| 0.4538 | 8.0 | 80 | 0.6643 | {'precision': 0.6775210084033614, 'recall': 0.7972805933250927, 'f1': 0.7325383304940376, 'number': 809} | {'precision': 0.23893805309734514, 'recall': 0.226890756302521, 'f1': 0.2327586206896552, 'number': 119} | {'precision': 0.7389830508474576, 'recall': 0.8187793427230047, 'f1': 0.7768374164810691, 'number': 1065} | 0.6878 | 0.7747 | 0.7286 | 0.8014 |
| 0.3958 | 9.0 | 90 | 0.6724 | {'precision': 0.7022222222222222, 'recall': 0.7812113720642769, 'f1': 0.7396138092451726, 'number': 809} | {'precision': 0.25757575757575757, 'recall': 0.2857142857142857, 'f1': 0.27091633466135456, 'number': 119} | {'precision': 0.7319932998324958, 'recall': 0.8206572769953052, 'f1': 0.7737937140327579, 'number': 1065} | 0.6918 | 0.7727 | 0.7300 | 0.7994 |
| 0.3902 | 10.0 | 100 | 0.6726 | {'precision': 0.6846071044133477, 'recall': 0.7861557478368356, 'f1': 0.7318757192174913, 'number': 809} | {'precision': 0.288135593220339, 'recall': 0.2857142857142857, 'f1': 0.2869198312236287, 'number': 119} | {'precision': 0.7652790079716564, 'recall': 0.8112676056338028, 'f1': 0.7876025524156791, 'number': 1065} | 0.7050 | 0.7697 | 0.7359 | 0.8080 |
| 0.3294 | 11.0 | 110 | 0.6827 | {'precision': 0.7018701870187019, 'recall': 0.788627935723115, 'f1': 0.7427240977881256, 'number': 809} | {'precision': 0.28888888888888886, 'recall': 0.3277310924369748, 'f1': 0.3070866141732283, 'number': 119} | {'precision': 0.7508561643835616, 'recall': 0.8234741784037559, 'f1': 0.7854903716972683, 'number': 1065} | 0.7025 | 0.7797 | 0.7391 | 0.8032 |
| 0.3124 | 12.0 | 120 | 0.6909 | {'precision': 0.6974697469746974, 'recall': 0.7836835599505563, 'f1': 0.7380675203725262, 'number': 809} | {'precision': 0.3125, 'recall': 0.33613445378151263, 'f1': 0.3238866396761134, 'number': 119} | {'precision': 0.771960958296362, 'recall': 0.8169014084507042, 'f1': 0.7937956204379562, 'number': 1065} | 0.7135 | 0.7747 | 0.7428 | 0.8047 |
| 0.2965 | 13.0 | 130 | 0.6986 | {'precision': 0.7002212389380531, 'recall': 0.7824474660074165, 'f1': 0.7390542907180385, 'number': 809} | {'precision': 0.3230769230769231, 'recall': 0.35294117647058826, 'f1': 0.3373493975903615, 'number': 119} | {'precision': 0.7712014134275619, 'recall': 0.819718309859155, 'f1': 0.7947200728265817, 'number': 1065} | 0.7147 | 0.7767 | 0.7444 | 0.8040 |
| 0.2676 | 14.0 | 140 | 0.7010 | {'precision': 0.7028824833702882, 'recall': 0.7836835599505563, 'f1': 0.7410870835768557, 'number': 809} | {'precision': 0.32575757575757575, 'recall': 0.36134453781512604, 'f1': 0.3426294820717131, 'number': 119} | {'precision': 0.7768888888888889, 'recall': 0.8206572769953052, 'f1': 0.7981735159817351, 'number': 1065} | 0.7184 | 0.7782 | 0.7471 | 0.8060 |
| 0.2747 | 15.0 | 150 | 0.7045 | {'precision': 0.7013274336283186, 'recall': 0.7836835599505563, 'f1': 0.7402218330414477, 'number': 809} | {'precision': 0.3111111111111111, 'recall': 0.35294117647058826, 'f1': 0.33070866141732286, 'number': 119} | {'precision': 0.773936170212766, 'recall': 0.819718309859155, 'f1': 0.796169630642955, 'number': 1065} | 0.7148 | 0.7772 | 0.7447 | 0.8045 |
### Framework versions
- Transformers 4.41.2
- Pytorch 2.3.0+cu121
- Datasets 2.19.2
- Tokenizers 0.19.1
|