End of training

README.md

@@ -0,0 +1,82 @@
+---
+library_name: transformers
+license: mit
+base_model: impira/layoutlm-document-qa
+tags:
+- generated_from_trainer
+datasets:
+- funsd
+model-index:
+- name: layoutlm-impira-funsd
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# layoutlm-impira-funsd
+
+This model is a fine-tuned version of [impira/layoutlm-document-qa](https://huggingface.co/impira/layoutlm-document-qa) on the funsd dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.9782
+- Answer: {'precision': 0.47514910536779326, 'recall': 0.5908529048207664, 'f1': 0.5267217630853994, 'number': 809}
+- Header: {'precision': 0.23809523809523808, 'recall': 0.25210084033613445, 'f1': 0.24489795918367344, 'number': 119}
+- Question: {'precision': 0.6262711864406779, 'recall': 0.6938967136150235, 'f1': 0.6583518930957684, 'number': 1065}
+- Overall Precision: 0.5394
+- Overall Recall: 0.6257
+- Overall F1: 0.5793
+- Overall Accuracy: 0.6785
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 3e-05
+- train_batch_size: 16
+- eval_batch_size: 8
+- seed: 42
+- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
+- lr_scheduler_type: linear
+- num_epochs: 15
+- mixed_precision_training: Native AMP
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss | Answer                                                                                                      | Header                                                                                                         | Question                                                                                                    | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
+|:-------------:|:-----:|:----:|:---------------:|:-----------------------------------------------------------------------------------------------------------:|:--------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:|
+| 1.7269        | 1.0   | 10   | 1.5271          | {'precision': 0.02695167286245353, 'recall': 0.03584672435105068, 'f1': 0.03076923076923077, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}                                                    | {'precision': 0.22638248847926268, 'recall': 0.36901408450704226, 'f1': 0.2806140664048554, 'number': 1065} | 0.1501            | 0.2117         | 0.1757     | 0.3784           |
+| 1.4601        | 2.0   | 20   | 1.3072          | {'precision': 0.11621021465581051, 'recall': 0.19406674907292953, 'f1': 0.14537037037037037, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}                                                    | {'precision': 0.31752232142857145, 'recall': 0.5342723004694836, 'f1': 0.39831991599579986, 'number': 1065} | 0.2310            | 0.3643         | 0.2827     | 0.4519           |
+| 1.2543        | 3.0   | 30   | 1.2448          | {'precision': 0.16467576791808874, 'recall': 0.23856613102595797, 'f1': 0.19485108531044926, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}                                                    | {'precision': 0.3877338877338877, 'recall': 0.7004694835680751, 'f1': 0.4991635998661759, 'number': 1065}   | 0.3033            | 0.4711         | 0.3690     | 0.4703           |
+| 1.1565        | 4.0   | 40   | 1.0455          | {'precision': 0.28119349005424954, 'recall': 0.38442521631644005, 'f1': 0.3248041775456919, 'number': 809}  | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}                                                    | {'precision': 0.44492574257425743, 'recall': 0.6751173708920187, 'f1': 0.5363670272286459, 'number': 1065}  | 0.3744            | 0.5168         | 0.4342     | 0.5789           |
+| 1.0464        | 5.0   | 50   | 0.9899          | {'precision': 0.30528846153846156, 'recall': 0.31396786155747836, 'f1': 0.30956733698964045, 'number': 809} | {'precision': 0.013888888888888888, 'recall': 0.008403361344537815, 'f1': 0.010471204188481676, 'number': 119} | {'precision': 0.45930232558139533, 'recall': 0.7417840375586855, 'f1': 0.5673249551166967, 'number': 1065}  | 0.3982            | 0.5243         | 0.4527     | 0.6244           |
+| 0.9482        | 6.0   | 60   | 0.9674          | {'precision': 0.3670774647887324, 'recall': 0.515451174289246, 'f1': 0.42879177377892036, 'number': 809}    | {'precision': 0.007194244604316547, 'recall': 0.008403361344537815, 'f1': 0.007751937984496125, 'number': 119} | {'precision': 0.5300429184549357, 'recall': 0.6957746478873239, 'f1': 0.6017052375152254, 'number': 1065}   | 0.4336            | 0.5815         | 0.4968     | 0.6382           |
+| 0.8653        | 7.0   | 70   | 1.0084          | {'precision': 0.41531322505800466, 'recall': 0.44252163164400493, 'f1': 0.42848593656493117, 'number': 809} | {'precision': 0.09933774834437085, 'recall': 0.12605042016806722, 'f1': 0.11111111111111109, 'number': 119}    | {'precision': 0.5230664857530529, 'recall': 0.723943661971831, 'f1': 0.6073257187869241, 'number': 1065}    | 0.4600            | 0.5740         | 0.5107     | 0.6432           |
+| 0.8103        | 8.0   | 80   | 0.9592          | {'precision': 0.4299153339604892, 'recall': 0.5648949320148331, 'f1': 0.48824786324786323, 'number': 809}   | {'precision': 0.1308411214953271, 'recall': 0.11764705882352941, 'f1': 0.12389380530973451, 'number': 119}     | {'precision': 0.5993511759935117, 'recall': 0.6938967136150235, 'f1': 0.6431679721496953, 'number': 1065}   | 0.5035            | 0.6071         | 0.5505     | 0.6516           |
+| 0.7159        | 9.0   | 90   | 0.9552          | {'precision': 0.4532803180914513, 'recall': 0.5636588380716935, 'f1': 0.5024793388429751, 'number': 809}    | {'precision': 0.1464968152866242, 'recall': 0.19327731092436976, 'f1': 0.16666666666666666, 'number': 119}     | {'precision': 0.6024590163934426, 'recall': 0.6901408450704225, 'f1': 0.6433260393873084, 'number': 1065}   | 0.5094            | 0.6091         | 0.5548     | 0.6544           |
+| 0.7067        | 10.0  | 100  | 0.9794          | {'precision': 0.4489795918367347, 'recall': 0.5982694684796045, 'f1': 0.5129835718071013, 'number': 809}    | {'precision': 0.19008264462809918, 'recall': 0.19327731092436976, 'f1': 0.19166666666666668, 'number': 119}    | {'precision': 0.6239168110918544, 'recall': 0.676056338028169, 'f1': 0.6489409643983776, 'number': 1065}    | 0.5215            | 0.6157         | 0.5647     | 0.6626           |
+| 0.6286        | 11.0  | 110  | 1.0066          | {'precision': 0.46477495107632094, 'recall': 0.5871446229913473, 'f1': 0.5188421627525941, 'number': 809}   | {'precision': 0.24603174603174602, 'recall': 0.2605042016806723, 'f1': 0.2530612244897959, 'number': 119}      | {'precision': 0.6328331862312445, 'recall': 0.6732394366197183, 'f1': 0.6524112829845314, 'number': 1065}   | 0.5362            | 0.6136         | 0.5723     | 0.6640           |
+| 0.6165        | 12.0  | 120  | 1.0739          | {'precision': 0.46348061316501354, 'recall': 0.6353522867737948, 'f1': 0.5359749739311783, 'number': 809}   | {'precision': 0.21929824561403508, 'recall': 0.21008403361344538, 'f1': 0.2145922746781116, 'number': 119}     | {'precision': 0.6371760500446828, 'recall': 0.6694835680751173, 'f1': 0.6529304029304029, 'number': 1065}   | 0.5346            | 0.6282         | 0.5776     | 0.6479           |
+| 0.5753        | 13.0  | 130  | 0.9666          | {'precision': 0.47213779128672745, 'recall': 0.5760197775030902, 'f1': 0.5189309576837415, 'number': 809}   | {'precision': 0.2366412213740458, 'recall': 0.2605042016806723, 'f1': 0.24800000000000003, 'number': 119}      | {'precision': 0.6175496688741722, 'recall': 0.7004694835680751, 'f1': 0.6564012318521778, 'number': 1065}   | 0.5344            | 0.6237         | 0.5756     | 0.6742           |
+| 0.5558        | 14.0  | 140  | 1.0031          | {'precision': 0.4807692307692308, 'recall': 0.6180469715698393, 'f1': 0.5408328826392644, 'number': 809}    | {'precision': 0.2689075630252101, 'recall': 0.2689075630252101, 'f1': 0.2689075630252101, 'number': 119}       | {'precision': 0.648, 'recall': 0.6845070422535211, 'f1': 0.6657534246575342, 'number': 1065}                | 0.5521            | 0.6327         | 0.5897     | 0.6773           |
+| 0.5368        | 15.0  | 150  | 0.9782          | {'precision': 0.47514910536779326, 'recall': 0.5908529048207664, 'f1': 0.5267217630853994, 'number': 809}   | {'precision': 0.23809523809523808, 'recall': 0.25210084033613445, 'f1': 0.24489795918367344, 'number': 119}    | {'precision': 0.6262711864406779, 'recall': 0.6938967136150235, 'f1': 0.6583518930957684, 'number': 1065}   | 0.5394            | 0.6257         | 0.5793     | 0.6785           |
+
+
+### Framework versions
+
+- Transformers 4.48.3
+- Pytorch 2.5.1+cu124
+- Datasets 3.5.0
+- Tokenizers 0.21.0

logs/events.out.tfevents.1745739094.b62a778f07fa.848.0

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ea2639ca67de1902962473fad789a67990e44ae88af7b0d615ca22bd78c3fc6b
-size 15360
+oid sha256:8c7c4b7185e6164098fef04248bcd5d2c21c20b5fecb4a691e2bfe6214b290a1
+size 16429

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:b62d96d67f8d3c31dbda1604801873344ce8d212fc0a4b4b410c130b18584183
+oid sha256:9580a5bf636da8587f43306c8fef7a73ff8dbb12d2322cfe73423d82b228960a
 size 511211780

preprocessor_config.json

@@ -0,0 +1,13 @@
+{
+  "apply_ocr": true,
+  "do_resize": true,
+  "image_processor_type": "LayoutLMv2ImageProcessor",
+  "ocr_lang": null,
+  "processor_class": "LayoutLMv2Processor",
+  "resample": 2,
+  "size": {
+    "height": 224,
+    "width": 224
+  },
+  "tesseract_config": ""
+}

special_tokens_map.json

@@ -0,0 +1,37 @@
+{
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[PAD]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,81 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [],
+  "apply_ocr": false,
+  "clean_up_tokenization_spaces": false,
+  "cls_token": "[CLS]",
+  "cls_token_box": [
+    0,
+    0,
+    0,
+    0
+  ],
+  "do_basic_tokenize": true,
+  "do_lower_case": true,
+  "extra_special_tokens": {},
+  "mask_token": "[MASK]",
+  "model_max_length": 512,
+  "never_split": null,
+  "only_label_first_subword": true,
+  "pad_token": "[PAD]",
+  "pad_token_box": [
+    0,
+    0,
+    0,
+    0
+  ],
+  "pad_token_label": -100,
+  "processor_class": "LayoutLMv2Processor",
+  "sep_token": "[SEP]",
+  "sep_token_box": [
+    1000,
+    1000,
+    1000,
+    1000
+  ],
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "LayoutLMv2Tokenizer",
+  "unk_token": "[UNK]"
+}