AlanCantoFTW's picture
Restore proven leaderboard YAML header and add Commercial Licensing section
71ed9e9 verified
|
Raw History Blame Contribute Delete
8.51 kB
---
library_name: transformers
pipeline_tag: image-text-to-text
gated: manual
extra_gated_prompt: "Access to ProKope-4B-Eagle-Eye weights is provided for academic evaluation, benchmarking, and commercial licensing inquiries. Please provide your affiliation and intended use case to request access."
extra_gated_fields:
Company or Institution: text
Corporate or Academic Email: text
Intended Evaluation Use Case: text
I agree not to redistribute the model weights outside approved evaluation harnesses: checkbox
datasets:
- allenai/olmOCR-bench
- llamaindex/ParseBench
- PaddlePaddle/Real5-OmniDocBench
- pulse-ai/PulseBench-Tab
- pulsia/fr-bench-pdf2md
tags:
- vision-language
- multimodal
- ocr
- document-parsing
- eval-results
- olmocr-bench
- parse-bench
- qwen3_5
- prokope
license: apache-2.0
language:
- en
- fr
- de
- es
- zh
- ja
- ko
- ru
- ar
model_name: ProKope-4B-Eagle-Eye
author: Alan Canto
creator: Alan Canto
---
<p align="center">
<img src="banner.jpg" alt="ProKope-4B-Eagle-Eye" width="100%" style="border-radius: 8px; margin-bottom: 16px;">
</p>
# ProKope-4B-Eagle-Eye
## Introduction
**ProKope-4B-Eagle-Eye** is a 4.2B-parameter multimodal vision-language model developed by [ProKope AI](https://huggingface.co/ProKope-AI) for document parsing, structured HTML table reconstruction (`rowspan`/`colspan`), mathematical LaTeX transcription, multi-column reading-order extraction, and chart interpretation.
The model uses the `Qwen3_5ForConditionalGeneration` architecture (`4.2B` parameters in `bfloat16`) with a 32K token context window, and loads directly in Hugging Face `transformers` and `vLLM` without custom remote code.
---
## Evaluation
Evaluated with deterministic greedy decoding (`do_sample=False`, `0-shot` Pass@1) across standard document OCR and table extraction benchmarks:
| Benchmark | Dataset | Split | Metric | ProKope-4B-Eagle-Eye | LightOnOCR-3-4B | Chandra-OCR-2 (4B) | olmOCR-2-7B |
| :--- | :--- | :---: | :--- | :---: | :---: | :---: | :---: |
| **PulseBench-Tab** | [`pulse-ai/PulseBench-Tab`](https://huggingface.co/datasets/pulse-ai/PulseBench-Tab) | `train (1,820)` | TEDS Similarity | **89.40%** | 84.40% | 78.90% | 70.50% |
| **olmOCR-Bench** | [`allenai/olmOCR-bench`](https://huggingface.co/datasets/allenai/olmOCR-bench) | `test (7 splits)` | Pass Rate | **87.06%** | 86.30% | 85.80% | 82.40% |
| **ParseBench** | [`llamaindex/ParseBench`](https://huggingface.co/datasets/llamaindex/ParseBench) | `test (5 pillars)` | Macro Average | **86.01%** | 75.10% | 70.10% | 43.30% |
| **fr-bench-pdf2md** | [`pulsia/fr-bench-pdf2md`](https://huggingface.co/datasets/pulsia/fr-bench-pdf2md) | `long_table` | Unit-Test Pass Rate | **85.11%** | 64.00% | 72.20% | 61.40% |
| **Real5-OmniDocBench** | [`PaddlePaddle/Real5-OmniDocBench`](https://huggingface.co/datasets/PaddlePaddle/Real5-OmniDocBench) | `test (5 scenarios)` | Overall Similarity | **76.61%** | — | — | — |
### `allenai/olmOCR-bench` Category Breakdown
| Category | ProKope-4B-Eagle-Eye | LightOnOCR-3-4B | Chandra-OCR-2 (4B) | olmOCR-2-7B |
| :--- | :---: | :---: | :---: | :---: |
| **ArXiv Math** (`arxiv_math`) | **95.56%** | 91.10% | 86.90% | 83.00% |
| **Old Scans Math** (`old_scans_math`) | **93.10%** | 89.10% | 89.10% | 82.30% |
| **Multi-Column** (`multi_column`) | **92.86%** | 85.60% | 82.10% | 83.70% |
| **Old Scans** (`old_scans`) | **85.19%** | 52.50% | 51.10% | 47.70% |
| **Tables** (`table_tests`) | 85.07% | 90.20% | **92.10%** | 84.90% |
| **Long Tiny Text** (`long_tiny_text`) | 79.41% | 93.20% | **93.70%** | 81.90% |
| **Headers & Footers** (`headers_footers`) | 78.26% | 89.20% | 91.40% | **96.10%** |
| **Overall** | **87.06%** | 86.30% | 85.80% | 82.40% |
### `llamaindex/ParseBench` Pillar Breakdown
| Pillar | Metric | ProKope-4B-Eagle-Eye | LightOnOCR-3-4B | Nemotron-3.5-Parse |
| :--- | :--- | :---: | :---: | :---: |
| **Table** (`table`) | TEDS Similarity | **93.95%** | 83.80% | 84.67% |
| **Layout** (`layout`) | Reading Order & Content | **90.46%** | 68.30% | 70.67% |
| **Text Content** (`text_content`) | 12-Rule Verbatim Match | 87.98% | **89.90%** | 87.96% |
| **Text Formatting** (`text_formatting`) | 11-Rule Hierarchy & Emphasis | **82.51%** | 67.60% | 68.28% |
| **Chart** (`chart`) | Label & Value Extraction | **75.17%** | 66.10% | 64.40% |
| **Overall (`mean`)** | 5-Pillar Macro Average | **86.01%** | 75.10% | 75.20% |
---
## Model Details
| Property | Value |
| :--- | :--- |
| **Architecture** | `Qwen3_5ForConditionalGeneration` |
| **Total Tensor Footprint** | `4.2B` combined parameters (`9.08 GB` in `bfloat16`) |
| **Context Length** | 32,768 tokens |
| **Recommended VRAM** | 12 GB+ for `bfloat16` inference |
| **Precision** | `torch.bfloat16` |
| **License** | Apache 2.0 (Weights gated under manual review) |
---
## Quickstart
### 1. Document & Table Extraction (`transformers`)
Requires `transformers`, `torch`, and `pillow`:
```python
import torch
from PIL import Image, ImageOps
from transformers import AutoProcessor, Qwen3_5ForConditionalGeneration
model_id = "ProKope-AI/ProKope-4B-Eagle-Eye"
processor = AutoProcessor.from_pretrained(model_id)
model = Qwen3_5ForConditionalGeneration.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto",
).eval()
image = Image.open("document_page.png").convert("RGB")
image = ImageOps.autocontrast(image, cutoff=1).resize((1024, 768), Image.Resampling.BILINEAR)
messages = [
{
"role": "user",
"content": [
{"type": "image", "image": image},
{
"type": "text",
"text": "[Visual Geometry: 1024x768px | Domain: complex_grid_table_html]\nExtract all tables and text on this document page into canonical HTML `<table>` structure and Markdown.",
},
],
}
]
prompt = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=[prompt], images=[image], return_tensors="pt").to(model.device)
inputs["pixel_values"] = inputs["pixel_values"].to(torch.bfloat16)
with torch.no_grad():
generated_ids = model.generate(**inputs, max_new_tokens=1408, do_sample=False, repetition_penalty=1.08)
response = processor.decode(generated_ids[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
print(response)
```
### 2. Serving with `vLLM`
```bash
vllm serve ProKope-AI/ProKope-4B-Eagle-Eye \
--dtype bfloat16 \
--max-model-len 16384 \
--limit-mm-per-prompt image=2
```
---
## Commercial Licensing & Enterprise Deployment
Access to `ProKope-4B-Eagle-Eye` weights is gated (`manual`) for benchmark verification, academic research, and commercial licensing evaluation. Production deployment is available through two options:
1. **Managed Parse API (`POST /v1/parse`)**: Pay-as-you-go multi-page PDF and image extraction returning clean Markdown, HTML tables, and structured JSON without managing GPU infrastructure.
2. **Self-Hosted Enterprise Node License**: Annual commercial license and turnkey Docker container (`vLLM` / `FastAPI`) for private VPC, HIPAA, SOC 2, and air-gapped on-premise deployments on a single 16GB–24GB GPU.
For API keys, enterprise trials, or commercial node licensing, contact **[prokope.model@gmail.com](mailto:prokope.model@gmail.com)** or submit the gated access form above with your corporate email and intended use case.
---
## Limitations & Intended Use
- **Intended Use**: Document OCR, structured Markdown conversion, HTML table reconstruction (`rowspan`/`colspan`), mathematical LaTeX transcription, and chart extraction.
- **Limitations**: Like all vision-language models, outputs on dense rasterized chart annotations (`ParseBench [chart]`: `75.17%`), sub-6pt footnote text (`olmOCR-bench [long_tiny_text]`: `79.41%`), or complex running watermarks (`olmOCR-bench [headers_footers]`: `78.26%`) may contain transcription errors. Verify critical financial or medical outputs independently.
---
## Contact & Citation
- **Organization**: [ProKope AI](https://huggingface.co/ProKope-AI)
- **Authors**: Alan Canto, Wesley Foreman
- **Contact**: [prokope.model@gmail.com](mailto:prokope.model@gmail.com)
```bibtex
@misc{canto2026prokope4beagleeye,
author = {Alan Canto and Wesley Foreman},
title = {ProKope-4B-Eagle-Eye: Multimodal Document Parsing and Table Reconstruction Model},
howpublished = {Hugging Face},
year = {2026},
url = {https://huggingface.co/ProKope-AI/ProKope-4B-Eagle-Eye}
}
```