File size: 9,424 Bytes
d16d35a | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 | ---
library_name: transformers
license: mit
base_model: microsoft/layoutlm-base-uncased
tags:
- generated_from_trainer
datasets:
- funsd
model-index:
- name: layoutlm-funsd
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# layoutlm-funsd
This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on the funsd dataset.
It achieves the following results on the evaluation set:
- Loss: 0.6896
- Answer: {'precision': 0.7152245345016429, 'recall': 0.8071693448702101, 'f1': 0.7584204413472706, 'number': 809}
- Header: {'precision': 0.3697478991596639, 'recall': 0.3697478991596639, 'f1': 0.3697478991596639, 'number': 119}
- Question: {'precision': 0.7833333333333333, 'recall': 0.8384976525821596, 'f1': 0.8099773242630386, 'number': 1065}
- Overall Precision: 0.7320
- Overall Recall: 0.7978
- Overall F1: 0.7635
- Overall Accuracy: 0.8098
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- num_epochs: 15
- mixed_precision_training: Native AMP
### Training results
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:-------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------:|:----------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:|
| 1.7932 | 1.0 | 10 | 1.6151 | {'precision': 0.013916500994035786, 'recall': 0.00865265760197775, 'f1': 0.010670731707317074, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.24816176470588236, 'recall': 0.1267605633802817, 'f1': 0.1678060907395898, 'number': 1065} | 0.1356 | 0.0712 | 0.0934 | 0.3264 |
| 1.4724 | 2.0 | 20 | 1.2863 | {'precision': 0.11666666666666667, 'recall': 0.1211372064276885, 'f1': 0.11885991510006065, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.3751733703190014, 'recall': 0.507981220657277, 'f1': 0.43159154367770247, 'number': 1065} | 0.2800 | 0.3206 | 0.2989 | 0.5749 |
| 1.1346 | 3.0 | 30 | 0.9582 | {'precision': 0.44279176201373, 'recall': 0.4783683559950556, 'f1': 0.45989304812834225, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.5849647611589663, 'recall': 0.7014084507042253, 'f1': 0.6379163108454312, 'number': 1065} | 0.5202 | 0.5690 | 0.5435 | 0.7037 |
| 0.8689 | 4.0 | 40 | 0.7730 | {'precision': 0.6069182389937107, 'recall': 0.715698393077874, 'f1': 0.6568349404424276, 'number': 809} | {'precision': 0.15384615384615385, 'recall': 0.06722689075630252, 'f1': 0.0935672514619883, 'number': 119} | {'precision': 0.6672519754170325, 'recall': 0.7136150234741784, 'f1': 0.6896551724137931, 'number': 1065} | 0.6280 | 0.6759 | 0.6510 | 0.7590 |
| 0.6834 | 5.0 | 50 | 0.6983 | {'precision': 0.6349206349206349, 'recall': 0.7416563658838071, 'f1': 0.6841505131128848, 'number': 809} | {'precision': 0.273972602739726, 'recall': 0.16806722689075632, 'f1': 0.20833333333333331, 'number': 119} | {'precision': 0.6741214057507987, 'recall': 0.7924882629107981, 'f1': 0.7285282693137678, 'number': 1065} | 0.6449 | 0.7346 | 0.6868 | 0.7813 |
| 0.5771 | 6.0 | 60 | 0.6775 | {'precision': 0.6438631790744467, 'recall': 0.7911001236093943, 'f1': 0.7099278979478648, 'number': 809} | {'precision': 0.3424657534246575, 'recall': 0.21008403361344538, 'f1': 0.2604166666666667, 'number': 119} | {'precision': 0.7335640138408305, 'recall': 0.7962441314553991, 'f1': 0.7636199909950472, 'number': 1065} | 0.6806 | 0.7592 | 0.7177 | 0.7871 |
| 0.5055 | 7.0 | 70 | 0.6602 | {'precision': 0.6920529801324503, 'recall': 0.7750309023485785, 'f1': 0.7311953352769679, 'number': 809} | {'precision': 0.3069306930693069, 'recall': 0.2605042016806723, 'f1': 0.28181818181818186, 'number': 119} | {'precision': 0.7590788308237378, 'recall': 0.8046948356807512, 'f1': 0.781221513217867, 'number': 1065} | 0.7093 | 0.7602 | 0.7338 | 0.7950 |
| 0.4549 | 8.0 | 80 | 0.6456 | {'precision': 0.6804670912951167, 'recall': 0.792336217552534, 'f1': 0.7321530553969159, 'number': 809} | {'precision': 0.2831858407079646, 'recall': 0.2689075630252101, 'f1': 0.27586206896551724, 'number': 119} | {'precision': 0.7497865072587532, 'recall': 0.8244131455399061, 'f1': 0.7853309481216457, 'number': 1065} | 0.6968 | 0.7782 | 0.7352 | 0.8069 |
| 0.3945 | 9.0 | 90 | 0.6484 | {'precision': 0.6906552094522019, 'recall': 0.7948084054388134, 'f1': 0.7390804597701149, 'number': 809} | {'precision': 0.30833333333333335, 'recall': 0.31092436974789917, 'f1': 0.3096234309623431, 'number': 119} | {'precision': 0.7660869565217391, 'recall': 0.8272300469483568, 'f1': 0.7954853273137698, 'number': 1065} | 0.7092 | 0.7832 | 0.7444 | 0.8067 |
| 0.3887 | 10.0 | 100 | 0.6674 | {'precision': 0.6968085106382979, 'recall': 0.8096415327564895, 'f1': 0.7489994282447112, 'number': 809} | {'precision': 0.31, 'recall': 0.2605042016806723, 'f1': 0.2831050228310502, 'number': 119} | {'precision': 0.790990990990991, 'recall': 0.8244131455399061, 'f1': 0.8073563218390805, 'number': 1065} | 0.7274 | 0.7847 | 0.7550 | 0.8115 |
| 0.3299 | 11.0 | 110 | 0.6748 | {'precision': 0.7125550660792952, 'recall': 0.799752781211372, 'f1': 0.7536400698893418, 'number': 809} | {'precision': 0.3305785123966942, 'recall': 0.33613445378151263, 'f1': 0.33333333333333337, 'number': 119} | {'precision': 0.7663230240549829, 'recall': 0.8375586854460094, 'f1': 0.800358905338717, 'number': 1065} | 0.7200 | 0.7923 | 0.7544 | 0.8053 |
| 0.3088 | 12.0 | 120 | 0.6757 | {'precision': 0.7155361050328227, 'recall': 0.8084054388133498, 'f1': 0.759141033081834, 'number': 809} | {'precision': 0.3904761904761905, 'recall': 0.3445378151260504, 'f1': 0.36607142857142855, 'number': 119} | {'precision': 0.7783641160949868, 'recall': 0.8309859154929577, 'f1': 0.8038147138964576, 'number': 1065} | 0.7328 | 0.7928 | 0.7616 | 0.8076 |
| 0.2922 | 13.0 | 130 | 0.6892 | {'precision': 0.7142857142857143, 'recall': 0.8096415327564895, 'f1': 0.7589803012746235, 'number': 809} | {'precision': 0.38461538461538464, 'recall': 0.37815126050420167, 'f1': 0.38135593220338987, 'number': 119} | {'precision': 0.7850133809099019, 'recall': 0.8262910798122066, 'f1': 0.8051235132662397, 'number': 1065} | 0.7332 | 0.7928 | 0.7618 | 0.8076 |
| 0.2692 | 14.0 | 140 | 0.6906 | {'precision': 0.7212389380530974, 'recall': 0.8059332509270705, 'f1': 0.7612375948628138, 'number': 809} | {'precision': 0.375, 'recall': 0.37815126050420167, 'f1': 0.37656903765690375, 'number': 119} | {'precision': 0.7841409691629956, 'recall': 0.8356807511737089, 'f1': 0.8090909090909091, 'number': 1065} | 0.7351 | 0.7963 | 0.7645 | 0.8087 |
| 0.2735 | 15.0 | 150 | 0.6896 | {'precision': 0.7152245345016429, 'recall': 0.8071693448702101, 'f1': 0.7584204413472706, 'number': 809} | {'precision': 0.3697478991596639, 'recall': 0.3697478991596639, 'f1': 0.3697478991596639, 'number': 119} | {'precision': 0.7833333333333333, 'recall': 0.8384976525821596, 'f1': 0.8099773242630386, 'number': 1065} | 0.7320 | 0.7978 | 0.7635 | 0.8098 |
### Framework versions
- Transformers 4.47.1
- Pytorch 2.5.1+cu121
- Datasets 3.2.0
- Tokenizers 0.21.0
|