File size: 9,263 Bytes
1ca0486 0788e37 1ca0486 0788e37 1ca0486 0788e37 1ca0486 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 |
---
tags:
- generated_from_trainer
datasets:
- funsd
model-index:
- name: layoutlm-funsd
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# layoutlm-funsd
This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on the funsd dataset.
It achieves the following results on the evaluation set:
- Loss: 0.6845
- Answer: {'precision': 0.6932907348242812, 'recall': 0.8046971569839307, 'f1': 0.7448512585812357, 'number': 809}
- Header: {'precision': 0.3220338983050847, 'recall': 0.31932773109243695, 'f1': 0.32067510548523204, 'number': 119}
- Question: {'precision': 0.7827225130890052, 'recall': 0.8422535211267606, 'f1': 0.8113975576662144, 'number': 1065}
- Overall Precision: 0.7199
- Overall Recall: 0.7958
- Overall F1: 0.7560
- Overall Accuracy: 0.8087
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 15
- mixed_precision_training: Native AMP
### Training results
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:-------------------------------------------------------------------------------------------------------------:|:----------------------------------------------------------------------------------------------------------:|:----------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:|
| 1.7948 | 1.0 | 10 | 1.5982 | {'precision': 0.019115890083632018, 'recall': 0.019777503090234856, 'f1': 0.01944106925880923, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.1559202813599062, 'recall': 0.12488262910798122, 'f1': 0.1386861313868613, 'number': 1065} | 0.0882 | 0.0748 | 0.0809 | 0.3666 |
| 1.4548 | 2.0 | 20 | 1.2137 | {'precision': 0.18571428571428572, 'recall': 0.19283065512978986, 'f1': 0.18920557913887204, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.5027844073190135, 'recall': 0.5934272300469483, 'f1': 0.5443583118001722, 'number': 1065} | 0.3758 | 0.3954 | 0.3853 | 0.6060 |
| 1.0759 | 3.0 | 30 | 0.9074 | {'precision': 0.45133689839572194, 'recall': 0.5216316440049443, 'f1': 0.48394495412844035, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.6371453138435081, 'recall': 0.6957746478873239, 'f1': 0.6651705565529622, 'number': 1065} | 0.5491 | 0.5835 | 0.5658 | 0.7138 |
| 0.818 | 4.0 | 40 | 0.7636 | {'precision': 0.601010101010101, 'recall': 0.7354758961681088, 'f1': 0.6614785992217899, 'number': 809} | {'precision': 0.22, 'recall': 0.09243697478991597, 'f1': 0.13017751479289943, 'number': 119} | {'precision': 0.6860670194003528, 'recall': 0.7305164319248826, 'f1': 0.707594361073215, 'number': 1065} | 0.6366 | 0.6944 | 0.6643 | 0.7580 |
| 0.6744 | 5.0 | 50 | 0.6948 | {'precision': 0.6172106824925816, 'recall': 0.7713226205191595, 'f1': 0.6857142857142857, 'number': 809} | {'precision': 0.2608695652173913, 'recall': 0.15126050420168066, 'f1': 0.19148936170212766, 'number': 119} | {'precision': 0.7063758389261745, 'recall': 0.7906103286384977, 'f1': 0.7461231723526807, 'number': 1065} | 0.6532 | 0.7446 | 0.6959 | 0.7803 |
| 0.5678 | 6.0 | 60 | 0.6772 | {'precision': 0.6684100418410042, 'recall': 0.7898640296662547, 'f1': 0.7240793201133144, 'number': 809} | {'precision': 0.32857142857142857, 'recall': 0.19327731092436976, 'f1': 0.2433862433862434, 'number': 119} | {'precision': 0.7155309033280507, 'recall': 0.847887323943662, 'f1': 0.7761065749892565, 'number': 1065} | 0.6840 | 0.7852 | 0.7311 | 0.7902 |
| 0.4886 | 7.0 | 70 | 0.6596 | {'precision': 0.6836518046709129, 'recall': 0.796044499381953, 'f1': 0.7355796687607081, 'number': 809} | {'precision': 0.30952380952380953, 'recall': 0.2184873949579832, 'f1': 0.2561576354679803, 'number': 119} | {'precision': 0.75, 'recall': 0.8422535211267606, 'f1': 0.793454223794781, 'number': 1065} | 0.7052 | 0.7863 | 0.7435 | 0.7931 |
| 0.4432 | 8.0 | 80 | 0.6525 | {'precision': 0.6792849631966351, 'recall': 0.7985166872682324, 'f1': 0.734090909090909, 'number': 809} | {'precision': 0.2736842105263158, 'recall': 0.2184873949579832, 'f1': 0.2429906542056075, 'number': 119} | {'precision': 0.7472984206151289, 'recall': 0.844131455399061, 'f1': 0.7927689594356261, 'number': 1065} | 0.6985 | 0.7883 | 0.7407 | 0.7965 |
| 0.3961 | 9.0 | 90 | 0.6515 | {'precision': 0.6940540540540541, 'recall': 0.7935723114956736, 'f1': 0.740484429065744, 'number': 809} | {'precision': 0.2803738317757009, 'recall': 0.25210084033613445, 'f1': 0.2654867256637167, 'number': 119} | {'precision': 0.7613344739093242, 'recall': 0.8356807511737089, 'f1': 0.7967770814682185, 'number': 1065} | 0.7097 | 0.7837 | 0.7449 | 0.8019 |
| 0.3531 | 10.0 | 100 | 0.6628 | {'precision': 0.697452229299363, 'recall': 0.8121137206427689, 'f1': 0.750428326670474, 'number': 809} | {'precision': 0.2962962962962963, 'recall': 0.2689075630252101, 'f1': 0.28193832599118945, 'number': 119} | {'precision': 0.7577276524644946, 'recall': 0.8516431924882629, 'f1': 0.801945181255526, 'number': 1065} | 0.7103 | 0.8008 | 0.7528 | 0.8034 |
| 0.3201 | 11.0 | 110 | 0.6678 | {'precision': 0.6915005246589717, 'recall': 0.8145859085290482, 'f1': 0.7480136208853576, 'number': 809} | {'precision': 0.2909090909090909, 'recall': 0.2689075630252101, 'f1': 0.2794759825327511, 'number': 119} | {'precision': 0.7679794520547946, 'recall': 0.8422535211267606, 'f1': 0.8034034930586654, 'number': 1065} | 0.7118 | 0.7968 | 0.7519 | 0.8071 |
| 0.3055 | 12.0 | 120 | 0.6760 | {'precision': 0.6869747899159664, 'recall': 0.8084054388133498, 'f1': 0.7427597955706984, 'number': 809} | {'precision': 0.296, 'recall': 0.31092436974789917, 'f1': 0.30327868852459017, 'number': 119} | {'precision': 0.7839506172839507, 'recall': 0.8347417840375587, 'f1': 0.8085493406093679, 'number': 1065} | 0.7146 | 0.7928 | 0.7517 | 0.8047 |
| 0.29 | 13.0 | 130 | 0.6844 | {'precision': 0.7013963480128894, 'recall': 0.8071693448702101, 'f1': 0.7505747126436783, 'number': 809} | {'precision': 0.28346456692913385, 'recall': 0.3025210084033613, 'f1': 0.2926829268292683, 'number': 119} | {'precision': 0.7771084337349398, 'recall': 0.847887323943662, 'f1': 0.8109564436461607, 'number': 1065} | 0.7171 | 0.7988 | 0.7558 | 0.8041 |
| 0.2797 | 14.0 | 140 | 0.6841 | {'precision': 0.6956055734190782, 'recall': 0.8022249690976514, 'f1': 0.7451205510907002, 'number': 809} | {'precision': 0.3064516129032258, 'recall': 0.31932773109243695, 'f1': 0.31275720164609055, 'number': 119} | {'precision': 0.7750865051903114, 'recall': 0.8413145539906103, 'f1': 0.8068437640702386, 'number': 1065} | 0.7153 | 0.7943 | 0.7527 | 0.8070 |
| 0.2733 | 15.0 | 150 | 0.6845 | {'precision': 0.6932907348242812, 'recall': 0.8046971569839307, 'f1': 0.7448512585812357, 'number': 809} | {'precision': 0.3220338983050847, 'recall': 0.31932773109243695, 'f1': 0.32067510548523204, 'number': 119} | {'precision': 0.7827225130890052, 'recall': 0.8422535211267606, 'f1': 0.8113975576662144, 'number': 1065} | 0.7199 | 0.7958 | 0.7560 | 0.8087 |
### Framework versions
- Transformers 4.26.1
- Pytorch 1.13.1+cu116
- Datasets 2.10.0
- Tokenizers 0.13.2
|