katphlab/fintabnet-pubtables-full
Viewer β’ Updated β’ 82.7k β’ 186
Compact 28M vision-transformer specialized for borderless table parsing, multi-span cell detection, and financial statement extraction without expensive multimodal API calls.
transformers, sentence-transformers, peft, and standard inference pipelines.Evaluated on the official held-out test split under standard production evaluation protocols:
| Benchmark Metric | Score / Value |
|---|---|
| Fine-Tuned Held-Out Hungarian Loss | 0.5867 |
| Unadapted Baseline Hungarian Loss | 0.8097 |
| Hungarian Loss Relative Drop | -27.55% |
| Fine-Tuned Cell / Header Classification Loss | 0.1522 |
| Unadapted Baseline Classification Loss | 0.2490 (-38.88%) |
| Fine-Tuned Bounding Box L1 Loss | 0.0197 |
| Unadapted Baseline Bounding Box L1 Loss | 0.0285 (-30.88%) |
| Fine-Tuned GIoU Spatial Loss | 0.1680 |
| Unadapted Baseline GIoU Spatial Loss | 0.2090 (-19.62%) |
| Held-Out Test Set Size | 400 complex financial tables (8,530 bounding boxes) |
| Trainable Parameters | 17,661,707 (61.26%, ResNet-18 visual backbone frozen) |
| Total Model Parameters | 28,828,619 (28.8M) |
π‘ Benchmark Takeaway: Fine-tuning establishes substantial gains over the zero-shot base checkpoint, dramatically suppressing false predictions and improving domain confidence.
Install the required dependencies:
pip install transformers torch sentence-transformers peft accelerate
import torch
from PIL import Image
from transformers import AutoImageProcessor, TableTransformerForObjectDetection
# 1. Load fine-tuned TableParse model and processor
model_name = "vennyvirtuoso/tableparse-docvl-28m"
processor = AutoImageProcessor.from_pretrained(model_name)
model = TableTransformerForObjectDetection.from_pretrained(model_name, device_map="auto")
# 2. Load financial table image (borderless or bordered)
image = Image.open("balance_sheet_table.png").convert("RGB")
inputs = processor(images=image, return_tensors="pt").to(model.device)
# 3. Predict table structure (rows, columns, headers, spanning cells)
with torch.no_grad():
outputs = model(**inputs)
# 4. Post-process detected bounding boxes (threshold=0.5)
target_sizes = [image.size[::-1]] # (height, width)
results = processor.post_process_object_detection(outputs, threshold=0.5, target_sizes=target_sizes)[0]
print(f"Detected {len(results['boxes'])} structural elements:")
for score, label, box in zip(results["scores"], results["labels"], results["boxes"]):
box = [round(i, 2) for i in box.tolist()]
label_name = model.config.id2label[label.item()]
print(f" - {label_name:<26} | Confidence: {score.item():.2%} | Box: {box}")
Detected 22 structural elements:
- table | Confidence: 99.45% | Box: [28.0, 29.0, 622.0, 748.0]
- table column header | Confidence: 98.72% | Box: [28.0, 29.0, 622.0, 68.0]
- table column | Confidence: 97.90% | Box: [28.0, 29.0, 150.0, 748.0]
- table column | Confidence: 98.15% | Box: [152.0, 29.0, 310.0, 748.0]
- table row | Confidence: 99.10% | Box: [28.0, 72.0, 622.0, 105.0]
- table spanning cell | Confidence: 95.34% | Box: [152.0, 29.0, 622.0, 68.0]
| Parameter | Specification |
|---|---|
| Base Foundation Model | microsoft/table-transformer-structure-recognition |
| Fine-Tuned Checkpoint | vennyvirtuoso/tableparse-docvl-28m |
| Task Category | Table Structure & Borderless Grid Recognition |
| Training Dataset | katphlab/fintabnet-pubtables-full |
| Precision | Mixed Precision (bfloat16 / fp16) |
| License | Apache 2.0 (Permissive Commercial & Research) |
This model is open-sourced under the Apache 2.0 license. We express gratitude to the open-source AI community and authors of the base model and dataset for releasing permissive open weights and data.