nickoo004/deepseek-ocr2-karakalpak-lora

DeepSeek-OCR-2 — Karakalpak OCR (LoRA)

LoRA fine-tune of deepseek-ai/DeepSeek-OCR-2 for Karakalpak-language OCR (Cyrillic script, including the extended glyphs ӊ, ў, қ, ғ, ҳ, ә). Trained on scanned pages of Karakalpak books.

This repository contains LoRA adapter weights only (~330 MB). You must load them on top of the base model.

Results

Evaluated on 100 held-out pages from books not seen during training (book-level split — no page leakage). Primary metric is CER (Character Error Rate), the standard metric for OCR; WER (Word Error Rate) is reported for reference. Lower is better.

Model	CER ↓	WER ↓	Mean latency (s)
DeepSeek-OCR-2 + LoRA (this model)	0.0556	0.1876	36.12
Tesseract 5 (rus)	0.1930	0.6529	1.50
DeepSeek-OCR-2 (base, no fine-tune)	0.4258	0.6987	17.45
GOT-OCR-2.0	1.1720	1.0701	14.28

LoRA fine-tuning reduced CER from 0.4258 to 0.0556 — an absolute drop of 0.3702 (86.9% relative improvement).

Note on Tesseract: the rus language pack lacks the Karakalpak-specific Cyrillic glyphs, which inflates its error rate — included as a classical non-neural baseline for context.

Training

• Base model: deepseek-ai/DeepSeek-OCR-2 (3B, deepseek_vl_v2 architecture)
• Method: LoRA (PEFT), rank 16, alpha 32, dropout 0.05
• Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• Trainable params: ~86M (2.48% of 3.48B)
• Vision encoder: frozen (SAM + ViT + projector run under no_grad)
• Prompt: <image>\nFree OCR. (plain-text targets, no layout/grounding)
• Hardware: 1× NVIDIA RTX 3090 (24 GB)
• Precision: bf16 + flash-attention-2 + gradient checkpointing
• Batch: effective 16 (bsz 1 × grad-accum 16)
• LR: 1e-4, cosine schedule, 3% warmup
• Train / val split: 9815 / 185 pages, grouped by book
• Best validation loss: 0.1610

Usage

import torch, tempfile
from transformers import AutoModel, AutoTokenizer
from peft import PeftModel

BASE = "deepseek-ai/DeepSeek-OCR-2"
ADAPTER = "nickoo004/deepseek-ocr2-karakalpak-lora"

tok = AutoTokenizer.from_pretrained(BASE, trust_remote_code=True)
model = AutoModel.from_pretrained(
    BASE, trust_remote_code=True, use_safetensors=True,
    _attn_implementation="flash_attention_2", torch_dtype=torch.bfloat16,
).cuda().eval()
model = PeftModel.from_pretrained(model, ADAPTER).eval()

# OCR a page
text = model.infer(
    tok, prompt="<image>\nFree OCR. ",
    image_file="page.png", output_path="/tmp/out",
    base_size=1024, image_size=768, crop_mode=True,
    save_results=False, eval_mode=True,
)
print(text)

Dataset

Private dataset: ocr-dataset — scanned Karakalpak book pages with transcribed plain-text labels (text_label).

Limitations

• Trained on plain-text transcription only; does not output layout, tables, or bounding boxes (use the base model for those).
• Optimized for printed Karakalpak Cyrillic; handwriting and other scripts are out of scope.
• Evaluated on 100 pages — broader evaluation recommended before production.

License

Apache-2.0, inherited from the base model.