|
|
--- |
|
|
language: en |
|
|
license: mit |
|
|
tags: |
|
|
- chess |
|
|
- ocr |
|
|
- handwritten-text-recognition |
|
|
- trocr |
|
|
- transformers |
|
|
- computer-vision |
|
|
- image-to-text |
|
|
datasets: |
|
|
- handwritten-chess-notation |
|
|
metrics: |
|
|
- accuracy |
|
|
- cer |
|
|
widget: |
|
|
- src: https://huggingface.co/datasets/mishig/sample_images/resolve/main/chess_example.png |
|
|
example_title: Chess Move Example |
|
|
pipeline_tag: image-to-text |
|
|
--- |
|
|
|
|
|
# Handwritten Chess Notation Recognition with TrOCR |
|
|
|
|
|
## Model Description |
|
|
|
|
|
This is a fine-tuned version of Microsoft's TrOCR-Large model, specifically trained to recognize handwritten chess notation from images of chess scoresheets. The model can accurately transcribe chess moves in Standard Algebraic Notation (SAN) from handwritten images. |
|
|
|
|
|
**Model Type:** Vision Encoder-Decoder |
|
|
**Architecture:** TrOCR (Transformer-based Optical Character Recognition) |
|
|
**Base Model:** `microsoft/trocr-large-handwritten` |
|
|
|
|
|
## Intended Uses & Limitations |
|
|
|
|
|
### Intended Use |
|
|
- Transcription of handwritten chess moves from scoresheet images |
|
|
- Digitization of historical chess games |
|
|
- Chess notation recognition in mobile apps |
|
|
- Educational tools for chess analysis |
|
|
|
|
|
### Limitations |
|
|
- Works best with clear handwriting |
|
|
- Trained specifically on chess notation (not general text) |
|
|
- May struggle with extremely cursive handwriting |
|
|
- Requires well-lit, focused images |
|
|
|
|
|
## Training Data |
|
|
|
|
|
The model was trained on a custom dataset of 13,731 handwritten chess move images with corresponding text labels in Standard Algebraic Notation. |
|
|
|
|
|
**Dataset Characteristics:** |
|
|
- Format: PNG images with move text annotations |
|
|
- Content: Chess moves in SAN format (e.g., "e4", "Nf3", "O-O", "Qxf7#") |
|
|
- Split: 88% training, 12% validation |
|
|
- Handwriting styles: Multiple variations |
|
|
|
|
|
## Training Procedure |
|
|
|
|
|
### Preprocessing |
|
|
Images were resized to 384x384 pixels and converted to RGB format. Text was tokenized with chess-specific vocabulary and padded/truncated to a maximum length of 16 tokens. |
|
|
|
|
|
### Training Hyperparameters |
|
|
- **Epochs:** 6 |
|
|
- **Batch Size:** 1 (with gradient accumulation of 8) |
|
|
- **Learning Rate:** 3e-5 |
|
|
- **Optimizer:** AdamW |
|
|
- **Weight Decay:** 0.01 |
|
|
- **Warmup Steps:** 200 |
|
|
- **Mixed Precision:** FP16 |
|
|
|
|
|
### Hardware |
|
|
- **GPU:** NVIDIA (8+ GB VRAM recommended) |
|
|
- **Training Time:** ~6 hours |
|
|
|
|
|
## Evaluation Results |
|
|
|
|
|
| Metric | Value | |
|
|
|--------|-------| |
|
|
| Accuracy | ~92% | |
|
|
| Character Error Rate (CER) | ~3% | |
|
|
| Inference Speed | ~100 ms/image | |
|
|
|
|
|
## How to Use |
|
|
|
|
|
### Direct Inference |
|
|
```python |
|
|
from transformers import TrOCRProcessor, VisionEncoderDecoderModel |
|
|
from PIL import Image |
|
|
import requests |
|
|
|
|
|
# Load model and processor |
|
|
model = VisionEncoderDecoderModel.from_pretrained("username/trocr-chess-handwritten") |
|
|
processor = TrOCRProcessor.from_pretrained("username/trocr-chess-handwritten") |
|
|
|
|
|
# Load and process image |
|
|
image = Image.open("chess_move.png").convert("RGB") |
|
|
pixel_values = processor(image, return_tensors="pt").pixel_values |
|
|
|
|
|
# Generate prediction |
|
|
generated_ids = model.generate(pixel_values) |
|
|
predicted_text = processor.batch_decode(generated_ids, skip_special_tokens=True)[0] |
|
|
print(f"Predicted move: {predicted_text}") |
