README.md

---
language:
  - uk
  - ru
license: apache-2.0
library_name: transformers
pipeline_tag: image-text-to-text
base_model: Qwen/Qwen3.5-4B
datasets:
  - UkrainianCatholicUniversity/rukopys
  - ai-forever/school_notebooks_RU
  - AntiplagiatCompany/HWR200
tags:
  - ocr
  - htr
  - handwritten-text-recognition
  - ukrainian
  - cyrillic
  - document-ai
  - layout-analysis
  - qwen3.5
---

# Rukopys-OCR-4B

**Rukopys-OCR-4B** is an open vision-language model for Ukrainian handwritten
document OCR. It detects document regions, classifies them, and returns their
transcriptions as structured JSON.

The model was created for the
[Handwritten to Data](https://www.kaggle.com/competitions/handwritten-to-data)
competition and was used in the **3rd-place final solution**. It is a full
fine-tune of [Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B).  
For training, evaluation, and ensemble details, see the
[competition writeup](https://www.kaggle.com/competitions/handwritten-to-data/writeups/short-writeup-for-public-2nd-place-solution).

## Output

```json
[
  {
    "bbox": [84, 107, 912, 168],
    "type": "handwritten",
    "text": "Приклад рукописного тексту"
  }
]
```

`bbox` is `[x1, y1, x2, y2]` in normalized `0..1000` coordinates. Valid types
are `handwritten`, `printed`, `formula`, `table`, `annotation`, `image`, and
`graph`. Formula text uses LaTeX; table text is pipe-separated; `image` and
`graph` use empty text.

## Inference

Use Transformers 5.8.1 or newer. The exact prompt used for training is included
below and should be kept unchanged.

### Transformers

```python
from PIL import Image
import torch
from transformers import AutoModelForMultimodalLM, AutoProcessor

MODEL_ID = "ebinan92/Rukopys-OCR-4B"
PROMPT = (
    "Detect every text region in this Ukrainian handwritten document and "
    "return a JSON array of regions. Each region has bbox (x1 y1 x2 y2 in "
    "0..1000 normalized image coordinates), type (handwritten | printed | "
    "formula | table | annotation | image | graph), and text (transcription; "
    "empty for image/graph; LaTeX for formula; pipe-separated for table)."
)

processor = AutoProcessor.from_pretrained(MODEL_ID)
model = AutoModelForMultimodalLM.from_pretrained(
    MODEL_ID, dtype=torch.bfloat16, device_map="auto"
)
image = Image.open("document.jpg").convert("RGB")
messages = [{
    "role": "user",
    "content": [{"type": "image"}, {"type": "text", "text": PROMPT}],
}]
text = processor.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True, enable_thinking=False
)
inputs = processor(text=[text], images=[image], return_tensors="pt").to(model.device)

with torch.inference_mode():
    output_ids = model.generate(**inputs, max_new_tokens=8192, do_sample=False)

new_tokens = output_ids[:, inputs["input_ids"].shape[1]:]
print(processor.batch_decode(new_tokens, skip_special_tokens=True)[0])
```

### vLLM

```python
from PIL import Image
from transformers import AutoProcessor
from vllm import LLM, SamplingParams

MODEL_ID = "ebinan92/Rukopys-OCR-4B"
PROMPT = (
    "Detect every text region in this Ukrainian handwritten document and "
    "return a JSON array of regions. Each region has bbox (x1 y1 x2 y2 in "
    "0..1000 normalized image coordinates), type (handwritten | printed | "
    "formula | table | annotation | image | graph), and text (transcription; "
    "empty for image/graph; LaTeX for formula; pipe-separated for table)."
)

processor = AutoProcessor.from_pretrained(MODEL_ID)
image = Image.open("document.jpg").convert("RGB")
messages = [{
    "role": "user",
    "content": [
        {"type": "image", "image": image},
        {"type": "text", "text": PROMPT},
    ],
}]
prompt = processor.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True, enable_thinking=False
)

factor = processor.image_processor.patch_size * processor.image_processor.merge_size
llm = LLM(
    model=MODEL_ID,
    dtype="bfloat16",
    max_model_len=16384,
    limit_mm_per_prompt={"image": 1},
    mm_processor_kwargs={
        "min_pixels": 256 * factor * factor,
        "max_pixels": 4096 * factor * factor,
    },
)
params = SamplingParams(max_tokens=8192, temperature=0.0)
outputs = llm.generate(
    [{"prompt": prompt, "multi_modal_data": {"image": image}}],
    sampling_params=params,
)
print(outputs[0].outputs[0].text)
```

## Training data and license

Training used RUKOPYS gold/silver data, external Cyrillic handwriting data,
and pseudo-labels, some of which were generated with Gemini
(`gemini-3-flash-preview`).

| Dataset | License |
|---|---|
| [RUKOPYS](https://huggingface.co/datasets/UkrainianCatholicUniversity/rukopys) | CC BY 4.0 |
| [Ukrainian Handwritten Text](https://www.kaggle.com/datasets/annyhnatiuk/ukrainian-handwritten-text) | CC BY-SA 4.0 |
| [school_notebooks_RU](https://huggingface.co/datasets/ai-forever/school_notebooks_RU) | MIT |
| [HWR200](https://huggingface.co/datasets/AntiplagiatCompany/HWR200) | Apache-2.0 |

The model weights are released under the **Apache License 2.0**. Training
datasets are not redistributed here and remain subject to their own licenses.

## Citation

```bibtex
@misc{ebinan2026rukopysocr4b,
  title        = {Rukopys-OCR-4B: Ukrainian Handwritten Document OCR},
  author       = {ebinan92},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/ebinan92/Rukopys-OCR-4B}}
}
```