| --- |
| library_name: transformers |
| license: apache-2.0 |
| base_model: google-bert/bert-base-multilingual-cased |
| tags: |
| - generated_from_trainer |
| - invoice-processing |
| - information-extraction |
| - czech-language |
| - synthetic-data |
| metrics: |
| - precision |
| - recall |
| - f1 |
| - accuracy |
| model-index: |
| - name: BERTInvoiceCzechR-V0 |
| results: [] |
| --- |
| |
| # BERTInvoiceCzechR (V0 – Synthetic Templates Only) |
|
|
| This model is a fine-tuned version of [google-bert/bert-base-multilingual-cased](https://huggingface.co/google-bert/bert-base-multilingual-cased) for the task of structured information extraction from Czech invoices. |
|
|
| It achieves the following results on the evaluation set: |
| - Loss: 0.3291 |
| - Precision: 0.5188 |
| - Recall: 0.6917 |
| - F1: 0.5929 |
| - Accuracy: 0.9335 |
|
|
| --- |
|
|
| ## Model description |
|
|
| BERTInvoiceCzechR (V0) is the baseline model in a multi-stage experimental pipeline focused on invoice understanding. |
|
|
| The model performs token-level classification to extract structured fields from invoice text, such as: |
| - supplier |
| - customer |
| - invoice number |
| - bank details |
| - totals |
| - dates |
|
|
| This version (V0) is trained **exclusively on synthetically generated invoices created from predefined templates**, without any layout randomization or real-world data. |
|
|
| --- |
|
|
| ## Training data |
|
|
| The dataset consists purely of: |
|
|
| - synthetically generated invoices |
| - fixed template structures |
| - controlled field placement and formatting |
|
|
| Characteristics: |
| - consistent layout across samples |
| - fully controlled annotations |
| - no noise or OCR artifacts |
| - no real invoice data |
| - added synthetic image augmentations |
| |
| This dataset represents the **simplest training scenario** in the pipeline and serves as a baseline for comparison with more complex data variants. |
|
|
| --- |
|
|
| ## Role in the pipeline |
|
|
| This model corresponds to: |
|
|
| **V0 – Synthetic template-based dataset only** |
|
|
| It is used as: |
| - a baseline for evaluating the impact of: |
| - layout variability |
| - synthetic-real hybrid data |
| - real annotated invoices |
| - a reference point for measuring generalization gap |
|
|
| --- |
|
|
| ## Intended uses |
|
|
| - Baseline model for document AI experiments |
| - Evaluation of synthetic data usefulness |
| - Comparison with more advanced dataset variants (V1–V3) |
| - Research in Czech invoice information extraction |
|
|
| --- |
|
|
| ## Limitations |
|
|
| - Strong dependency on template structure |
| - May have poor generalization to: |
| - unseen layouts |
| - real-world invoices |
| - noisy OCR outputs |
| - Does not capture layout variability |
| - Trained only on clean synthetic data |
|
|
| --- |
|
|
| ## Training procedure |
|
|
| ### Training hyperparameters |
|
|
| The following hyperparameters were used during training: |
| - learning_rate: 1e-05 |
| - train_batch_size: 16 |
| - eval_batch_size: 2 |
| - seed: 42 |
| - optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments |
| - lr_scheduler_type: linear |
| - lr_scheduler_warmup_steps: 0.1 |
| - num_epochs: 10 |
| - mixed_precision_training: Native AMP |
|
|
| --- |
|
|
| ### Training results |
|
|
| | Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1 | Accuracy | |
| |:-------------:|:-----:|:----:|:---------------:|:---------:|:------:|:------:|:--------:| |
| | No log | 1.0 | 87 | 0.3944 | 0.1965 | 0.2233 | 0.2091 | 0.8997 | |
| | No log | 2.0 | 174 | 0.2951 | 0.4152 | 0.4517 | 0.4327 | 0.9241 | |
| | No log | 3.0 | 261 | 0.2896 | 0.4790 | 0.5810 | 0.5251 | 0.9314 | |
| | No log | 4.0 | 348 | 0.3295 | 0.4549 | 0.6443 | 0.5333 | 0.9226 | |
| | No log | 5.0 | 435 | 0.3249 | 0.4908 | 0.6866 | 0.5724 | 0.9281 | |
| | 0.3757 | 6.0 | 522 | 0.3615 | 0.4646 | 0.6827 | 0.5529 | 0.9216 | |
| | 0.3757 | 7.0 | 609 | 0.3376 | 0.4913 | 0.6579 | 0.5625 | 0.9299 | |
| | 0.3757 | 8.0 | 696 | 0.3290 | 0.5194 | 0.6924 | 0.5935 | 0.9336 | |
| | 0.3757 | 9.0 | 783 | 0.3604 | 0.4906 | 0.6858 | 0.5720 | 0.9279 | |
| | 0.3757 | 10.0 | 870 | 0.3515 | 0.5011 | 0.6944 | 0.5821 | 0.9296 | |
|
|
| --- |
|
|
| ## Framework versions |
|
|
| - Transformers 5.0.0 |
| - PyTorch 2.10.0+cu128 |
| - Datasets 4.0.0 |
| - Tokenizers 0.22.2 |
