| --- |
| language: en |
| license: mit |
| tags: |
| - layoutlmv3 |
| - invoice-parsing |
| - document-understanding |
| - token-classification |
| - ner |
| - pdf |
| base_model: microsoft/layoutlmv3-base |
| pipeline_tag: token-classification |
| --- |
| |
| # PDF Invoice Parser — Fine-tuned LayoutLMv3 |
|
|
| A fine-tuned [LayoutLMv3](https://huggingface.co/microsoft/layoutlmv3-base) model for named entity recognition (NER) on PDF invoices. It extracts structured fields such as invoice number, dates, vendor/customer details, and financial totals directly from document pages using text, layout (bounding boxes), and visual features. |
|
|
| ## Model Details |
|
|
| - **Base model:** `microsoft/layoutlmv3-base` |
| - **Architecture:** `LayoutLMv3ForTokenClassification` |
| - **Task:** Token classification (NER) |
| - **Fine-tuned on:** Labeled PDF invoice pages |
|
|
| ## Labels |
|
|
| | Label | Description | |
| |---|---| |
| | `B/I-INVOICE_NUM` | Invoice number | |
| | `B/I-INVOICE_DATE` | Invoice date | |
| | `B/I-DUE_DATE` | Payment due date | |
| | `B/I-VENDOR_NAME` | Vendor / seller name | |
| | `B/I-VENDOR_ADDR` | Vendor address | |
| | `B/I-CUST_NAME` | Customer / buyer name | |
| | `B/I-CUST_ADDR` | Customer address | |
| | `B/I-TOTAL` | Total amount | |
| | `B/I-SUBTOTAL` | Subtotal amount | |
| | `B/I-TAX` | Tax amount | |
| | `O` | Outside / no entity | |
|
|
| ## Quick Start |
|
|
| ```bash |
| pip install transformers torch Pillow |
| ``` |
|
|
| ```python |
| from transformers import LayoutLMv3Processor, LayoutLMv3ForTokenClassification |
| import torch |
| from PIL import Image |
| |
| processor = LayoutLMv3Processor.from_pretrained("Kapilydv6/layoutlmv3-invoice-parser", apply_ocr=False) |
| model = LayoutLMv3ForTokenClassification.from_pretrained("Kapilydv6/layoutlmv3-invoice-parser") |
| model.eval() |
| |
| # words and boxes come from your OCR tool (e.g. pytesseract) |
| encoding = processor( |
| image, # PIL.Image of the invoice page |
| words, # list of word strings |
| boxes=boxes, # list of [x0, y0, x1, y1] normalized to 0–1000 |
| return_tensors="pt", |
| truncation=True, |
| padding="max_length", |
| max_length=512, |
| ) |
| |
| with torch.no_grad(): |
| outputs = model(**encoding) |
| |
| predictions = outputs.logits.argmax(-1).squeeze().tolist() |
| id2label = model.config.id2label |
| predicted_labels = [id2label[p] for p in predictions] |
| ``` |
|
|
| ## Full Pipeline (PDF → JSON) |
|
|
| ```python |
| from invoice_parser import InvoiceParser |
| |
| parser = InvoiceParser(strategy="finetuned") |
| result = parser.parse("invoice.pdf") |
| print(result.to_json()) |
| ``` |
|
|
| ## Output Format |
|
|
| ```json |
| { |
| "invoice_number": "INV-2024-0042", |
| "invoice_date": "March 15, 2024", |
| "due_date": "April 15, 2024", |
| "vendor_name": "Acme Corp", |
| "vendor_address": "123 Business St, City", |
| "customer_name": "Client LLC", |
| "customer_address": "456 Client Ave, Town", |
| "subtotal": 1200.00, |
| "tax": 216.00, |
| "total": 1416.00 |
| } |
| ``` |
|
|
| ## Extraction Strategies (invoice_parser.py) |
| |
| | Strategy | Speed | Accuracy | Best For | |
| |---|---|---|---| |
| | `pdfplumber` | Fast | Good | Digital/typed PDFs | |
| | `ocr` | Moderate | Good | Scanned PDFs | |
| | `finetuned` | Moderate | Very Good | Complex layouts (this model) | |
| | `claude` | Moderate | Excellent | Any PDF (needs API key) | |
| |
| ## Training |
| |
| Fine-tuned using `train_model.py` on labeled invoice annotations produced by `label_invoices.py`. |
| |
| ```bash |
| python train_model.py --annotations annotations/ --output trained_model/ --epochs 15 |
| ``` |
| |
| ## License |
| |
| MIT |
| |
