sherif1313/3arab-OCR-v1

💜 Github   |   🤗 Hugging Face   |   📚 Cookbooks
🖥️ Demo

3arabOCR-v1

Arabic Vision-Language OCR for Printed Text and Historical Manuscripts

3arabOCR-v1 is a multimodal Arabic vision-language OCR model with approximately 3.3 billion parameters, designed primarily for recognizing Arabic text from images.

The model is specifically fine-tuned and further trained for:

• Printed Arabic books
• Historical Arabic manuscripts
• Historical documents
• Arabic poetry
• Documents containing varied handwriting styles

🚀 Quick Overview

Property	Value
Model	3arabOCR-v1
Parameters	~3.3B
Architecture	Vision-Language + MoE
Language	Arabic
Vision Encoder	CLIP + SAM
Language Model	DeepSeekV2 MoE
Experts	64
Context	Up to 32,768 tokens
Primary Task	Arabic OCR
Printed CER	0.80%
Manuscript CER	1.07%
License	Apache-2.0

🌟 Key Features

High-Accuracy Arabic OCR

The model achieves a CER below 1% on the evaluated printed Arabic dataset.

Historical Manuscript Recognition

The model has been specifically trained on historical Arabic material, including documents, manuscripts, deeds, correspondence, and other archival material.

Multimodal Architecture

The model combines:

• CLIP visual representations
• SAM spatial representations
• A trainable multimodal projector
• DeepSeekV2-based MoE language generation

Long Context

The model supports contexts of up to approximately:

• 32,768 tokens

This makes it suitable for processing larger pages and long OCR outputs.

Page Tiling

Large pages can be automatically divided into multiple crops to improve recognition of small or dense text.

📊 Benchmark Results

The following results are from the project's internal evaluation.

The evaluation used four Arabic text categories with 30 images per category.

Category	CER	WER
Printed Arabic Books	0.80%	3.22%
Arabic Handwriting	1.07%	4.72%

Interpretation

The results indicate strong performance on the evaluated printed and historical Arabic datasets.

Very difficult handwriting remains an active area of development.

📐 Evaluation Methodology

CER and WER were calculated using the jiwer library.

The generated OCR output was evaluated against the corresponding reference transcription for each page.

Character Error Rate

CER measures character-level transcription errors:

CER = Character Errors / Reference Characters

Word Error Rate

WER measures word-level transcription errors:

WER = Word Errors / Reference Words

CER is particularly useful for OCR because a single character error can change the meaning of an Arabic word.

Installation

Requires Python 3.10+ (tested on 3.12):

# ── Transformers stack ──
transformers>=4.57.1
tokenizers>=0.20
safetensors>=0.4
huggingface_hub>=0.24

# ── معالجة الصور والمدخلات ──
pillow>=10.0
numpy>=1.26

# ── مساعدات الاستدلال ──
einops>=0.7
tqdm>=4.66
sentencepiece>=0.1.99
protobuf>=4.25

bitsandbytes   #  is **required** — this release ships pre-quantized 4-bit weights. The repository includes custom modeling code, loaded via `trust_remote_code=True`.

PDFs

import os
import tempfile
import fitz
import torch
from transformers import AutoModel, AutoTokenizer

MODEL_NAME = "sherif1313/3arab-OCR-v1"
PDF_PATH = "/BOOK/BOOK.pdf"
OUTPUT_DIR = "./3/output"
DPI = 200
PROMPT = "<image>document parsing."

print("Loading model...")

tokenizer = AutoTokenizer.from_pretrained(
    MODEL_NAME, trust_remote_code=True
)

model = AutoModel.from_pretrained(
    MODEL_NAME,
    trust_remote_code=True,
    use_safetensors=True,
    torch_dtype=torch.bfloat16,
).eval().cuda()

print("Model loaded.")


def pdf_to_images(pdf_path, dpi=DPI):
    if not os.path.exists(pdf_path):
        raise FileNotFoundError(pdf_path)

    doc = fitz.open(pdf_path)
    temp_dir = tempfile.mkdtemp(prefix="ocr_")
    scale = dpi / 72
    matrix = fitz.Matrix(scale, scale)
    images = []

    for i, page in enumerate(doc, 1):
        path = os.path.join(temp_dir, f"page_{i:04d}.png")
        page.get_pixmap(matrix=matrix, alpha=False).save(path)
        images.append(path)
        print(f"Converted {i}/{len(doc)}")

    doc.close()
    return images


def ocr_pages(images):
    os.makedirs(OUTPUT_DIR, exist_ok=True)

    for i, image in enumerate(images, 1):
        print(f"OCR page {i}/{len(images)}")

        try:
            model.infer(
                tokenizer,
                prompt=PROMPT,
                image_file=image,
                output_path=OUTPUT_DIR,
                base_size=1024,
                image_size=640,
                crop_mode=True,
                max_length=8192,
                no_repeat_ngram_size=35,
                ngram_window=1024,
                save_results=True,
            )
        except Exception as e:
            print(f"Error on page {i}: {e}")


images = pdf_to_images(PDF_PATH)
ocr_pages(images)

print(f"Done. Results: {OUTPUT_DIR}")

Visualization

Memory requirements

Approximate inference VRAM for the 4-bit release: 3–6 GB, depending on image resolution, number of crops, output length, and batch size. The full-precision model requires ~8 GB. device_map="auto" can spread the model across multiple GPUs.

Known limitations

• Difficult handwriting and heavily damaged manuscripts.
• general multilingual OCR — other languages are untested.
• Does not reconstruct page layout (columns, tables, margins, heading positions).
• Complex tables are not supported.

Performance can degrade substantially when:

• Writing is highly connected
• Ink is faded
• Pages are damaged

Roadmap

• Full-precision (bf16) release
• Reinforced multi-page / document-level inference
• Layout understanding: text regions, tables, columns, headings, bounding boxes
• Structured Arabic document output: Image → Layout Detection → OCR → Structured Document
• In the next version, I plan to further improve the model by integrating PaddleOCR-based layout detection and training with layout-aware data containing text regions, bounding boxes, and their spatial relationships.

Therefore, the model may recognize the text correctly while not perfectly preserving:

• Columns
• Tables
• Complex document structures

Future development may include stronger document-layout understanding.

The architecture also provides model.infer_multi(...) for joint multi-page parsing (prompt: "<image>Multi page parsing.", image_size=1024). In this release, multi-page behavior has not been reinforced by fine-tuning, so single-page processing is the reliable path — improved multi-page support is on the roadmap.

A potential pipeline is:

Image
  │
  ▼
Layout Detection
  │
  ├── Text Regions
  ├── Tables
  ├── Columns
  ├── Headings
  └── Paragraphs
  │
  ▼
OCR
  │
  ▼
Structured Arabic Document

Planned capabilities

• Page-layout detection
• Text-region detection
• Bounding-box prediction
• Paragraph reconstruction
• Heading recognition
• List recognition
• Table recognition
• Multi-column reconstruction
• Spatial relationship understanding
• Improved document structure preservation

🧠 Model Components

Component	Description
CLIP ViT	Extracts visual and semantic image features
SAM	Provides spatial and image-region information
Projector	Aligns visual features with the language-model representation space
DeepSeekV2	Performs language processing and generation
MoE	Uses a 64-expert mixture-of-experts architecture

🏗️ Architecture

                         Input Image
                              │
                              ▼
                    ┌──────────────────┐
                    │   CLIP Vision    │
                    └──────────────────┘
                              │
                              ├──────────┐
                              │          │
                              ▼          ▼
                       Visual Features  SAM
                              │          │
                              └────┬─────┘
                                   │
                                   ▼
                            Multimodal
                             Projector
                                   │
                                   ▼
                            Visual Tokens
                                   │
                                   ▼
                       DeepSeekV2 MoE LLM
                                   │
                         ┌─────────┴─────────┐
                         │   64 Experts      │
                         └─────────┬─────────┘
                                   │
                                   ▼
                              Arabic Text

Interpretation

This confirms that 3arabOCR-v1 is not merely a renamed checkpoint: the language component was substantially re-trained on Arabic data while the vision encoder remained largely unchanged.

• LLM / MoE accounts for almost all of the meaningful change. More than 82% of its individual weights were modified, and the deep reconstruction of the language head (lm_head.weight) confirms that the Arabic language modeling capacity was re-learned.
• CLIP / Vision, SAM, and Projector changed very broadly across tensors but with negligible magnitude (all below 0.01% by the Global Ratio metric). The visual backbone is therefore essentially preserved.
• The only structural difference between the two checkpoints is a single non-trainable buffer, model.vision_model.embeddings.position_ids, which does not affect behavior.

Parameter-shape compatibility:

Component	Changed Tensors	Changed Elements	Global Ratio	Per-Tensor Avg	Per-Element Avg
CLIP / Vision	80.95%	7.04%	0.0103%	0.0057%	1.34%
SAM	84.36%	4.08%	0.0043%	0.0023%	1.24%
Projector	100.00%	5.66%	0.0089%	0.0062%	1.03%
LLM / MoE	98.93%	82.54%	2.53%	5.17%	86.99%
Other (view_seperator)	100.00%	20.94%	0.5221%	0.5221%	48.93%
ALL	96.01%	73.37%	2.38%	4.27%	76.69%

All compared parameter shapes are compatible, while the actual parameter values were modified through additional training.

How to Read These Numbers

• Changed Tensors % — the fraction of tensors whose values differ according to torch.equal. It only indicates whether a tensor changed, not by how much.
• Changed Elements % — the fraction of individual scalar weights that changed.
• Global Ratio — the most robust measure of the magnitude of change. This is the value closest to the model card.
• Per-Element Avg — sensitive to near-zero weights and should not be interpreted as the magnitude of the change.

However, token accuracy should not be interpreted as final OCR accuracy.

For OCR evaluation, CER and WER are the more relevant metrics.

🖼️ Long-Page Processing

Large pages can be split automatically into multiple image crops.

The current training/inference workflow supports up to approximately:

14 crops per page

depending on the page and crop workload.

This approach is useful for:

• High-resolution books
• Dense Arabic pages
• Small fonts
• Historical documents
• Large manuscript pages

📂 Repository Structure

.
├── model.safetensors
├── modeling_arabocr.py
├── modeling_deepseekv2.py
├── deepencoder.py
├── configuration_deepseek_v2.py
├── conversation.py
├── requirements.txt
└── README.md

Main Files

File	Description
modeling_arabocr.py	Main OCR model architecture
modeling_deepseekv2.py	DeepSeekV2 MoE implementation
deepencoder.py	CLIP + SAM visual encoder
configuration_deepseek_v2.py	DeepSeekV2 configuration
conversation.py	Conversation/input formatting
requirements.txt	Python dependencies

Acknowledgements

• baidu/Unlimited-OCR
• The DeepSeek-OCR-2, CLIP, and SAM communities

The training methodology and accompanying project code were developed by the project author.

Citation

@misc{3arabocr_v1,
  title  = {3arab-OCR-v1: An Arabic Vision-Language OCR Model},
  author = {sherif1313},
  year   = {2026},
  url    = {https://huggingface.co/sherif1313/3arab-OCR-v1}
}

License

apache-2.0.