File size: 2,329 Bytes
4aaceba 38970db 874a801 38970db 874a801 38970db 58a71d1 38970db 58a71d1 38970db 874a801 38970db 874a801 38970db 874a801 38970db 874a801 38970db 874a801 38970db 874a801 38970db 874a801 38970db 874a801 38970db 874a801 38970db | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 | ---
library_name: transformers
pipeline_tag: image-to-text
base_model: microsoft/trocr-base-printed
tags:
- trocr
- vision-encoder-decoder
- image-to-text
- ocr
- captcha
- math-captcha
- synthetic-data
---
# OCR Match Captcha
A TrOCR model fine-tuned to recognize short mathematical captcha
expressions such as `26+7=?` and `45-6=?`.
<table border="0" cellspacing="24" cellpadding="12">
<tbody>
<tr>
<td style="border:none;padding:18px;"><img src="assets/captcha-sample-2.png" alt="Handwritten captcha" width="290" height="90"></td>
<td style="border:none;padding:18px;"><img src="assets/captcha-sample-3.png" alt="Clean captcha" width="290" height="90"></td>
</tr>
<tr>
<td style="border:none;padding:18px;"><img src="assets/captcha-sample-4.png" alt="Distorted captcha" width="290" height="90"></td>
<td style="border:none;padding:18px;"><img src="assets/captcha-sample-1.png" alt="Math captcha" width="290" height="90"></td>
</tr>
</tbody>
</table>
## Model details
- **Architecture:** Vision Encoder-Decoder / TrOCR
- **Base model:** `microsoft/trocr-base-printed`
- **Task:** Image-to-text OCR
- **Target geometry:** 130x30 RGB images
- **Output:** Mathematical expressions without spaces
## Training configuration
| Parameter | Value |
|---|---:|
| Epochs | 5 |
| Learning rate | 5e-6 |
| Batch size | 8 |
| Gradient accumulation | 2 |
| Operator-token weight | 3.0 |
| Generation beams | 4 |
| Maximum output length | 32 |
## Usage
```python
from PIL import Image
from transformers import TrOCRProcessor, VisionEncoderDecoderModel
repo_id = "arkhabbazan/ocr-match-captcha"
processor = TrOCRProcessor.from_pretrained(repo_id, token=True)
model = VisionEncoderDecoderModel.from_pretrained(repo_id, token=True)
image = Image.open("captcha.jpeg").convert("RGB")
pixel_values = processor(images=image, return_tensors="pt").pixel_values
generated_ids = model.generate(pixel_values, num_beams=4, max_length=32)
prediction = processor.batch_decode(
generated_ids, skip_special_tokens=True
)[0]
print(prediction.replace(" ", ""))
```
## Limitations
- Training data is primarily synthetic.
- Unseen fonts and layouts may reduce accuracy.
- The model is intended for short expressions, not document OCR.
- Usage must be authorized and compliant with applicable policies.
|