---
license: apache-2.0
language:
- ru
- en
base_model:
- raxtemur/trocr-base-ru
pipeline_tag: image-to-text
tags:
- htr
- trocr
- image-to-text
- transformers
datasets:
- CherryJam/ru-dialectological-stackmix
- CherryJam/ru-dialectological-fonts-aug
---

# Russian Dialectic HTR using TrOCR

The TrOCR-base-ru model was fine-tuned on synthetic datasets generated using [StackMix](https://huggingface.co/datasets/CherryJam/ru-dialectological-stackmix) method
and based on various [fonts](https://huggingface.co/datasets/CherryJam/ru-dialectological-fonts-aug).

For more information, check out the [GitHub repository](https://github.com/DialecticalHTR/RuDialect-HTR).

## Model description

TrOCR-base-ru was fine-tuned for Handwritten Russian Text Recognition in dialectological cards.
The model was trained for a total of 20 epochs with a batch size of 16 using dual NVIDIA T4 GPUs. The fine-tuning process took approximately 28 hours.

# Example Usage

```python
# Load libraries
from transformers import TrOCRProcessor, VisionEncoderDecoderModel
import matplotlib.pyplot as plt
from PIL import Image


# Load image
img_path = 'path/to/image'
image = Image.open(img_path).convert("RGB")

# Load model and processor
model_name = "Daniil-Domino/trocr-base-ru-dialectic-stackmix"
processor = TrOCRProcessor.from_pretrained(model_name)
model = VisionEncoderDecoderModel.from_pretrained(model_name)

# Preprocess and run inference
pixel_values = processor(images=image, return_tensors="pt").pixel_values
generated_ids = model.generate(pixel_values)
generated_text = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]

# Output result
print(generated_text)

# Display input image
plt.axis("off")
plt.imshow(image)
plt.show()
```

# Metrics
Below are the key evaluation metrics on the validation set:
- **CER**: 14.87 %
- **WER**: 45.71 %
- **Accuracy**: 55.28 %