Add model card

README.md

@@ -0,0 +1,202 @@
+---
+language: en
+license: apache-2.0
+tags:
+  - document-ai
+  - layoutlmv3
+  - token-classification
+  - receipt-extraction
+  - invoice-extraction
+  - base-model
+datasets:
+  - custom
+metrics:
+  - f1
+  - precision
+  - recall
+---
+
+# layoutlmv3-receipt-invoice
+
+LayoutLMv3 model initialized for receipt and invoice field extraction.
+
+## Model Status
+
+⚠️ **This is an initialized base model** - not yet fine-tuned on custom data.
+
+- **Base Model**: `microsoft/layoutlmv3-base`
+- **Status**: Ready for deployment and fine-tuning
+- **Custom Labels**: Configured for receipt/invoice field extraction
+
+## Intended Use
+
+This model is configured to extract the following fields from receipts and invoices:
+
+### Supported Fields
+
+[
+  "O",
+  "B-MerchantName",
+  "I-MerchantName",
+  "B-MerchantAddress",
+  "I-MerchantAddress",
+  "B-TransactionDate",
+  "I-TransactionDate",
+  "B-Currency",
+  "I-Currency",
+  "B-Total",
+  "I-Total",
+  "B-TotalTax",
+  "I-TotalTax",
+  "B-InvoiceNumber",
+  "I-InvoiceNumber",
+  "B-Subtotal",
+  "I-Subtotal",
+  "B-LineItems",
+  "I-LineItems"
+]
+
+## Training Status
+
+This repository contains:
+- ✅ Base LayoutLMv3 architecture
+- ✅ Custom label configuration for receipts/invoices
+- ⏳ **Not yet fine-tuned** - using pre-trained weights from `microsoft/layoutlmv3-base`
+
+### Training the Model
+
+To fine-tune this model on your custom data:
+
+```bash
+# On RunPod GPU pod or local machine with GPU
+python main.py --mode train --push-to-hub --version v1.0
+```
+
+This will:
+1. Train on your labeled receipt/invoice data
+2. Update this repository with fine-tuned weights
+3. Tag the trained version (e.g., v1.0, v1.1, etc.)
+
+## Usage
+
+### Local Inference
+
+```python
+from transformers import LayoutLMv3ForTokenClassification, LayoutLMv3Processor
+from PIL import Image
+
+# Load model and processor
+model = LayoutLMv3ForTokenClassification.from_pretrained("mkdigitalgmbh/runpo-LayoutLM3-Invoice-Receipt")
+processor = LayoutLMv3Processor.from_pretrained("mkdigitalgmbh/runpo-LayoutLM3-Invoice-Receipt", apply_ocr=False)
+
+# Prepare inputs (you need OCR results: words and bounding boxes)
+image = Image.open("receipt.jpg").convert("RGB")
+words = ["STORE", "NAME", "Total:", "$10.99"]
+boxes = [[10, 10, 100, 30], [110, 10, 200, 30], [10, 50, 80, 70], [90, 50, 150, 70]]
+
+# Normalize boxes to 0-1000 range
+width, height = image.size
+normalized_boxes = [[int(1000*x0/width), int(1000*y0/height),
+                      int(1000*x1/width), int(1000*y1/height)] for x0,y0,x1,y1 in boxes]
+
+encoding = processor(image, words, boxes=normalized_boxes, return_tensors="pt")
+outputs = model(**encoding)
+predictions = outputs.logits.argmax(-1)
+```
+
+### RunPod Serverless Deployment
+
+This model is designed for deployment on RunPod Serverless:
+
+1. **Build and push Docker image:**
+   ```bash
+   cd deployment/runpod/LayoutLMv3
+   python deploy.py --action deploy
+   ```
+
+2. **Create RunPod endpoint:**
+   - Docker Image: `registry.hf.space/your-username/layoutlmv3-inference:latest`
+   - Environment Variables:
+     - `HF_REPO_ID=mkdigitalgmbh/runpo-LayoutLM3-Invoice-Receipt`
+     - `HF_TOKEN=<your-token>`
+     - `MODEL_VERSION=main` (or specific version tag after training)
+
+## Model Architecture
+
+- **Base**: microsoft/layoutlmv3-base
+- **Task**: Token Classification
+- **Input**: Image + Words + Bounding Boxes
+- **Output**: Field labels (IOB tagging scheme)
+- **Number of Labels**: 19
+
+## Label Schema
+
+The model uses IOB (Inside-Outside-Beginning) tagging:
+
+- **O**: Outside any field
+- **B-FieldName**: Beginning of a field
+- **I-FieldName**: Inside/continuation of a field
+
+### Example
+
+```
+Text:        ["Total:", "$", "10", ".", "99"]
+Labels:      ["B-Total", "I-Total", "I-Total", "I-Total", "I-Total"]
+Extracted:   Total: "$ 10 . 99"
+```
+
+## Version History
+
+| Version | Date | Description | Status |
+|---------|------|-------------|--------|
+| main | 2025-11-13 | Initialized with base model + custom labels | Base (not trained) |
+
+After training, versions will be tagged (v1.0, v1.1, etc.).
+
+## Training Configuration
+
+When training is performed, the following configuration will be used:
+
+```python
+{
+  "model_name": "microsoft/layoutlmv3-base",
+  "learning_rate": 5e-05,
+  "batch_size": 4,
+  "num_epochs": 20,
+  "warmup_steps": 500,
+  "max_length": 512,
+  "validation_split": 0.2,
+  "random_seed": 42,
+  "gradient_accumulation_steps": 2,
+  "eval_steps": 100,
+  "save_steps": 500,
+  "logging_steps": 50
+}
+```
+
+## Citation
+
+```bibtex
+@misc{layoutlmv3-receipt-invoice,
+  author = {MK Digital GmbH},
+  title = {LayoutLMv3 Receipt/Invoice Field Extraction},
+  year = {2025},
+  publisher = {Hugging Face},
+  howpublished = {\url{https://huggingface.co/mkdigitalgmbh/runpo-LayoutLM3-Invoice-Receipt}}
+}
+
+@article{huang2022layoutlmv3,
+  title={LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking},
+  author={Huang, Yupan and Lv, Tengchao and Cui, Lei and Lu, Yutong and Wei, Furu},
+  journal={arXiv preprint arXiv:2204.08387},
+  year={2022}
+}
+```
+
+## License
+
+Apache 2.0
+
+## Contact
+
+For questions or issues, please open an issue in the repository.