End of training

README.md

@@ -0,0 +1,79 @@
+---
+tags:
+- generated_from_trainer
+datasets:
+- funsd
+model-index:
+- name: layoutlm-funsd
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# layoutlm-funsd
+
+This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on the funsd dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.6924
+- Answer: {'precision': 0.6991525423728814, 'recall': 0.8158220024721878, 'f1': 0.7529948659440959, 'number': 809}
+- Header: {'precision': 0.3253968253968254, 'recall': 0.3445378151260504, 'f1': 0.33469387755102037, 'number': 119}
+- Question: {'precision': 0.7807250221043325, 'recall': 0.8291079812206573, 'f1': 0.8041894353369764, 'number': 1065}
+- Overall Precision: 0.7197
+- Overall Recall: 0.7948
+- Overall F1: 0.7554
+- Overall Accuracy: 0.8040
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 3e-05
+- train_batch_size: 16
+- eval_batch_size: 8
+- seed: 42
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- num_epochs: 15
+- mixed_precision_training: Native AMP
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss | Answer                                                                                                       | Header                                                                                                      | Question                                                                                                      | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
+|:-------------:|:-----:|:----:|:---------------:|:------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:|
+| 1.8027        | 1.0   | 10   | 1.6152          | {'precision': 0.0103359173126615, 'recall': 0.004944375772558714, 'f1': 0.006688963210702341, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}                                                 | {'precision': 0.18232044198895028, 'recall': 0.061971830985915494, 'f1': 0.09250175192711983, 'number': 1065} | 0.0935            | 0.0351         | 0.0511     | 0.3249           |
+| 1.4899        | 2.0   | 20   | 1.2751          | {'precision': 0.18495297805642633, 'recall': 0.21878862793572312, 'f1': 0.20045300113250283, 'number': 809}  | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}                                                 | {'precision': 0.4306969459671104, 'recall': 0.5164319248826291, 'f1': 0.46968403074295473, 'number': 1065}    | 0.3254            | 0.3648         | 0.3440     | 0.5899           |
+| 1.1133        | 3.0   | 30   | 0.9514          | {'precision': 0.4911937377690802, 'recall': 0.6205191594561187, 'f1': 0.5483342435827417, 'number': 809}     | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}                                                 | {'precision': 0.5651144435674822, 'recall': 0.672300469483568, 'f1': 0.614065180102916, 'number': 1065}       | 0.5314            | 0.6111         | 0.5685     | 0.6982           |
+| 0.8513        | 4.0   | 40   | 0.8326          | {'precision': 0.5850746268656717, 'recall': 0.7268232385661311, 'f1': 0.6482910694597575, 'number': 809}     | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}                                                 | {'precision': 0.6844003606853021, 'recall': 0.7126760563380282, 'f1': 0.6982520699172033, 'number': 1065}     | 0.6248            | 0.6759         | 0.6493     | 0.7384           |
+| 0.7153        | 5.0   | 50   | 0.7422          | {'precision': 0.6159274193548387, 'recall': 0.7552533992583437, 'f1': 0.6785119378123265, 'number': 809}     | {'precision': 0.08695652173913043, 'recall': 0.05042016806722689, 'f1': 0.06382978723404256, 'number': 119} | {'precision': 0.6826051112943117, 'recall': 0.7774647887323943, 'f1': 0.726953467954346, 'number': 1065}      | 0.6354            | 0.7250         | 0.6773     | 0.7734           |
+| 0.5972        | 6.0   | 60   | 0.7031          | {'precision': 0.6330645161290323, 'recall': 0.7762669962917181, 'f1': 0.6973903387007219, 'number': 809}     | {'precision': 0.15492957746478872, 'recall': 0.09243697478991597, 'f1': 0.11578947368421053, 'number': 119} | {'precision': 0.6802189210320563, 'recall': 0.8169014084507042, 'f1': 0.742320819112628, 'number': 1065}      | 0.6443            | 0.7572         | 0.6962     | 0.7836           |
+| 0.5209        | 7.0   | 70   | 0.6902          | {'precision': 0.6597510373443983, 'recall': 0.7861557478368356, 'f1': 0.7174280879864636, 'number': 809}     | {'precision': 0.2755102040816326, 'recall': 0.226890756302521, 'f1': 0.2488479262672811, 'number': 119}     | {'precision': 0.7128463476070529, 'recall': 0.7971830985915493, 'f1': 0.7526595744680851, 'number': 1065}     | 0.6711            | 0.7587         | 0.7122     | 0.7902           |
+| 0.4673        | 8.0   | 80   | 0.6693          | {'precision': 0.6649642492339122, 'recall': 0.8046971569839307, 'f1': 0.7281879194630874, 'number': 809}     | {'precision': 0.28, 'recall': 0.23529411764705882, 'f1': 0.2557077625570776, 'number': 119}                 | {'precision': 0.7322314049586777, 'recall': 0.831924882629108, 'f1': 0.7789010989010988, 'number': 1065}      | 0.6837            | 0.7852         | 0.7310     | 0.7965           |
+| 0.4151        | 9.0   | 90   | 0.6684          | {'precision': 0.6839323467230444, 'recall': 0.799752781211372, 'f1': 0.7373219373219373, 'number': 809}      | {'precision': 0.30327868852459017, 'recall': 0.31092436974789917, 'f1': 0.3070539419087137, 'number': 119}  | {'precision': 0.7363560033585222, 'recall': 0.8234741784037559, 'f1': 0.7774822695035462, 'number': 1065}     | 0.6910            | 0.7832         | 0.7342     | 0.8017           |
+| 0.3689        | 10.0  | 100  | 0.6742          | {'precision': 0.6954643628509719, 'recall': 0.796044499381953, 'f1': 0.7423631123919308, 'number': 809}      | {'precision': 0.3157894736842105, 'recall': 0.3025210084033613, 'f1': 0.30901287553648066, 'number': 119}   | {'precision': 0.7483221476510067, 'recall': 0.8375586854460094, 'f1': 0.7904297740363314, 'number': 1065}     | 0.7043            | 0.7888         | 0.7441     | 0.8000           |
+| 0.3327        | 11.0  | 110  | 0.6861          | {'precision': 0.6843198338525441, 'recall': 0.8145859085290482, 'f1': 0.7437923250564334, 'number': 809}     | {'precision': 0.32456140350877194, 'recall': 0.31092436974789917, 'f1': 0.31759656652360513, 'number': 119} | {'precision': 0.7709790209790209, 'recall': 0.828169014084507, 'f1': 0.7985513807152557, 'number': 1065}      | 0.7105            | 0.7918         | 0.7489     | 0.8031           |
+| 0.3167        | 12.0  | 120  | 0.6912          | {'precision': 0.6896186440677966, 'recall': 0.8046971569839307, 'f1': 0.7427267541357673, 'number': 809}     | {'precision': 0.3416666666666667, 'recall': 0.3445378151260504, 'f1': 0.34309623430962344, 'number': 119}   | {'precision': 0.7727272727272727, 'recall': 0.8300469483568075, 'f1': 0.8003621548211861, 'number': 1065}     | 0.7138            | 0.7908         | 0.7503     | 0.8013           |
+| 0.3012        | 13.0  | 130  | 0.6878          | {'precision': 0.7015086206896551, 'recall': 0.8046971569839307, 'f1': 0.7495682210708117, 'number': 809}     | {'precision': 0.33064516129032256, 'recall': 0.3445378151260504, 'f1': 0.33744855967078186, 'number': 119}  | {'precision': 0.7694974003466204, 'recall': 0.8338028169014085, 'f1': 0.8003605227579991, 'number': 1065}     | 0.7162            | 0.7928         | 0.7526     | 0.8073           |
+| 0.2882        | 14.0  | 140  | 0.6930          | {'precision': 0.6997885835095138, 'recall': 0.8182941903584673, 'f1': 0.7544159544159544, 'number': 809}     | {'precision': 0.33064516129032256, 'recall': 0.3445378151260504, 'f1': 0.33744855967078186, 'number': 119}  | {'precision': 0.7793468667255075, 'recall': 0.8291079812206573, 'f1': 0.8034576888080072, 'number': 1065}     | 0.7199            | 0.7958         | 0.7560     | 0.8024           |
+| 0.2811        | 15.0  | 150  | 0.6924          | {'precision': 0.6991525423728814, 'recall': 0.8158220024721878, 'f1': 0.7529948659440959, 'number': 809}     | {'precision': 0.3253968253968254, 'recall': 0.3445378151260504, 'f1': 0.33469387755102037, 'number': 119}   | {'precision': 0.7807250221043325, 'recall': 0.8291079812206573, 'f1': 0.8041894353369764, 'number': 1065}     | 0.7197            | 0.7948         | 0.7554     | 0.8040           |
+
+
+### Framework versions
+
+- Transformers 4.26.1
+- Pytorch 1.13.1+cu116
+- Datasets 2.9.1.dev0
+- Tokenizers 0.13.2

logs/events.out.tfevents.1676245847.9620fdf4cf0a.6598.0

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:954f7dd1a4c0ff5db389d47ce2091b27581c072f2c2421fa11beea90c37b2fce
-size 9937
+oid sha256:671cf70d0c2376da16096e48f77156bbe8f85001756a770ee7acc681707c254e
+size 14221

preprocessor_config.json

@@ -0,0 +1,14 @@
+{
+  "apply_ocr": true,
+  "do_resize": true,
+  "feature_extractor_type": "LayoutLMv2FeatureExtractor",
+  "image_processor_type": "LayoutLMv2ImageProcessor",
+  "ocr_lang": null,
+  "processor_class": "LayoutLMv2Processor",
+  "resample": 2,
+  "size": {
+    "height": 224,
+    "width": 224
+  },
+  "tesseract_config": ""
+}

pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:a7a71765bdc02a9446c0ed6e03330cd44655d8236f63721f3e7ebf60eae5ef22
+oid sha256:7c47b26826d5e7b32525f40e5c20982f1dfddb0ba59fbb598b871213a38e7b46
 size 450608389

special_tokens_map.json

@@ -0,0 +1,7 @@
+{
+  "cls_token": "[CLS]",
+  "mask_token": "[MASK]",
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "unk_token": "[UNK]"
+}

tokenizer_config.json

@@ -0,0 +1,39 @@
+{
+  "additional_special_tokens": null,
+  "apply_ocr": false,
+  "cls_token": "[CLS]",
+  "cls_token_box": [
+    0,
+    0,
+    0,
+    0
+  ],
+  "do_basic_tokenize": true,
+  "do_lower_case": true,
+  "mask_token": "[MASK]",
+  "model_max_length": 512,
+  "name_or_path": "microsoft/layoutlmv2-base-uncased",
+  "never_split": null,
+  "only_label_first_subword": true,
+  "pad_token": "[PAD]",
+  "pad_token_box": [
+    0,
+    0,
+    0,
+    0
+  ],
+  "pad_token_label": -100,
+  "processor_class": "LayoutLMv2Processor",
+  "sep_token": "[SEP]",
+  "sep_token_box": [
+    1000,
+    1000,
+    1000,
+    1000
+  ],
+  "special_tokens_map_file": null,
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "LayoutLMv2Tokenizer",
+  "unk_token": "[UNK]"
+}