README.md

---
license: mit
language:
  - en
tags:
  - captcha
  - ocr
  - crnn
  - ctc
  - image-to-text
  - tensorflow
  - keras
task_categories:
  - image-to-text
datasets:
  - ayoubkirouane/captcha
  - ThangaTharun/captchaimages
  - yuxi5/text-captcha-data-clean
  - AvinashRicky/CaptchaOCR-500K
  - cybertruck32489/captcha_90k
  - yusuf802/captcha_dataset
  - lumasik/captcha-25k
---

# 🔐 CRNN-CTC Captcha Solver

### Convolutional Recurrent Neural Network · Real-Data Fine-Tuned

[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Python 3.10+](https://img.shields.io/badge/Python-3.10+-blue.svg)](https://www.python.org/)
[![TensorFlow 2.21](https://img.shields.io/badge/TensorFlow-2.21-orange.svg)](https://www.tensorflow.org/)

Alphanumeric CAPTCHA recognition using a deep **CRNN + CTC** architecture trained on
**275,000 real-world labeled CAPTCHAs** from 7 HuggingFace datasets, then refined over a
full weekend of augmented training on an NVIDIA A100. Achieves **90.08% whole-CAPTCHA
sequence accuracy** on the held-out real test set.

![image](preview_all_types.png)
---

## 📋 Model Details

| Property | Value |
|---|---|
| **Task** | Alphanumeric CAPTCHA Recognition (OCR) |
| **Architecture** | CRNN — 6-block CNN + 2× Bidirectional LSTM + Dense |
| **Input** | RGB image `(64 × 200 × 3)`, float32 `[0, 1]` |
| **Output** | Character sequence, length 1–8 |
| **Vocabulary** | `0-9`, `a-z`, `A-Z` — 62 characters + CTC blank |
| **Loss** | Connectionist Temporal Classification (CTC) |
| **Parameters** | 10,049,535 (~38.3 MB) |
| **Framework** | TensorFlow 2.21 / Keras 3 |
| **Training hardware** | NVIDIA A100 80 GB |

---

## 📊 Performance

### Overall (held-out real test set, n = 5,502)

| Metric | Score |
|---|---|
| **Sequence accuracy** (whole CAPTCHA correct) | **90.08 %** |
| **Character accuracy** | **96.14 %** |
| CTC loss | 1.10 |

### Per-dataset breakdown

| Dataset | n (test) | Char acc | Seq acc |
|---|---|---|---|
| cybertruck32489/captcha_90k | 1,224 | 99.9 % | **99.7 %** |
| ayoubkirouane/captcha | 192 | 99.9 % | **99.5 %** |
| yuxi5/text-captcha-data-clean | 1,171 | 93.6 % | 90.9 % |
| AvinashRicky/CaptchaOCR-500K | 1,215 | 97.8 % | 88.7 % |
| lumasik/captcha-25k | 502 | 96.0 % | 86.5 % |
| yusuf802/captcha_dataset | 1,196 | 92.5 % | 80.8 % |
| ThangaTharun/captchaimages | 2 | 100.0 % | 100.0 % |

**Total params: 10,049,535 — Model size: 38.3 MB**

---

## 📦 Training Datasets

| Dataset | Label field | Images used | Notes |
|---|---|---|---|
| [ayoubkirouane/captcha](https://huggingface.co/datasets/ayoubkirouane/captcha) | `solution` | 10,000 | clean, parquet |
| [ThangaTharun/captchaimages](https://huggingface.co/datasets/ThangaTharun/captchaimages) | `output` | 100 | small, noisy |
| [yuxi5/text-captcha-data-clean](https://huggingface.co/datasets/yuxi5/text-captcha-data-clean) | `label` | 60,000 | capped at 60k |
| [AvinashRicky/CaptchaOCR-500K](https://huggingface.co/datasets/AvinashRicky/CaptchaOCR-500K) | `text` | 60,000 | capped at 60k |
| [cybertruck32489/captcha_90k](https://huggingface.co/datasets/cybertruck32489/captcha_90k) | `solve` | 60,000 | capped at 60k |
| [yusuf802/captcha_dataset](https://huggingface.co/datasets/yusuf802/captcha_dataset) | `label` | 60,000 | capped at 60k |
| [lumasik/captcha-25k](https://huggingface.co/datasets/lumasik/captcha-25k) | `label` | 25,000 | all images used |
| **Synthetic (17 generator types)** | generated | mixed in at 30% | custom Python generators |

**Total real images preprocessed:** 275,100 → 264,096 train / 5,502 val / 5,502 test

All images were resized to **64 × 200** with aspect-preserving letterboxing (pad with
median border color).

---

## ⚙️ Training Details

### Two-stage process

**Stage 1 — Synthetic pre-training**
Training from scratch on 17 procedural CAPTCHA types (custom Python generators) covering
different fonts, noise levels, distortions, and color palettes.

**Stage 2 — Real-data fine-tuning + augmented weekend run**

- Initialized from the synthetic checkpoint.
- Fine-tuned on 264k real images for 40 epochs (LR 1e-3 → 1e-5 cosine).
- Extended with a full weekend run (386 epochs, ~60h on A100) combining:
  - **30% synthetic** mixed into each batch for robustness.
  - **GPU augmentation:** `RandomRotation(±2°)`, `RandomTranslation`, `RandomZoom(6%)`, `RandomContrast(25%)`, brightness jitter ±0.08, Gaussian noise σ=0.03.
  - **Weak-source oversampling:** datasets below 90% test seq were oversampled up to ×2 proportionally.

### Hyperparameters

| Parameter | Value |
|---|---|
| Optimizer | Adam |
| Peak LR | 5 × 10⁻⁴ |
| Min LR | 1 × 10⁻⁶ |
| LR schedule | Warmup + Cosine Decay |
| Batch size | 128 |
| Dropout | 0.25 |
| Gradient clip norm | 5.0 |
| Max sequence length | 8 characters |

---

## 🚀 Usage

### Install dependencies

```bash
pip install tensorflow pillow numpy
```

### Predict from an image

```python
import numpy as np
from PIL import Image
import tensorflow as tf

# ── helpers ──────────────────────────────────────────────────────────────────
CHARS       = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"
IDX_TO_CHAR = {i: c for i, c in enumerate(CHARS)}
BLANK_IDX   = 62
IMG_H, IMG_W = 64, 200

def preprocess(pil_img: Image.Image) -> np.ndarray:
    """Aspect-preserving resize to 64×200, pad with median border color."""
    img = pil_img.convert("RGB")
    w, h = img.size
    new_w = int(round(w * IMG_H / h))
    img = img.resize((new_w, IMG_H), Image.LANCZOS)
    if new_w >= IMG_W:
        img = img.resize((IMG_W, IMG_H), Image.LANCZOS)
        return np.array(img, dtype=np.float32) / 255.0
    canvas = np.array(img, dtype=np.float32)
    border_color = np.median(
        np.concatenate([canvas[0], canvas[-1],
                        canvas[:, 0], canvas[:, -1]], axis=0), axis=0
    )
    pad_l = (IMG_W - new_w) // 2
    result = np.full((IMG_H, IMG_W, 3), border_color, dtype=np.float32)
    result[:, pad_l:pad_l + new_w] = canvas
    return result / 255.0

def decode_ctc(logits: np.ndarray) -> str:
    """Greedy CTC decode — collapse repeats, remove blanks."""
    indices = np.argmax(logits, axis=-1)       # (T,)
    chars, prev = [], -1
    for idx in indices:
        if idx != prev and idx != BLANK_IDX:
            chars.append(IDX_TO_CHAR.get(int(idx), ""))
        prev = idx
    return "".join(chars)

# ── load model ───────────────────────────────────────────────────────────────
# Clone the repo or download the weights file, then:
from model import build_crnn_model   # from this repository

model = build_crnn_model()
model.load_weights("weekend_best.weights.h5")

# ── run inference ─────────────────────────────────────────────────────────────
img    = Image.open("captcha.png")
tensor = preprocess(img)[np.newaxis]           # (1, 64, 200, 3)
logits = model(tensor, training=False).numpy() # (1, 50, 63)
print("Prediction:", decode_ctc(logits[0]))
```

### Batch inference

```python
images = [preprocess(Image.open(p)) for p in image_paths]
batch  = np.stack(images)                      # (N, 64, 200, 3)
logits = model(batch, training=False).numpy()  # (N, 50, 63)
preds  = [decode_ctc(l) for l in logits]
```

---

## 🖼️ Preprocessing

Input images go through **aspect-preserving letterboxing**:

1. Resize height to 64 px, keeping aspect ratio.
2. If the resulting width ≥ 200 px, squeeze to 200 px.
3. Otherwise, pad left and right with the **median border color** of the image to reach 200 px.
4. Normalize to `float32 [0, 1]`.

This ensures the character shapes are never stretched, which is critical for
distinguishing similar characters (e.g. `O` / `0`, `l` / `1`).

---

## ⚖️ License

This model is released under the **MIT License**. You are free to use, copy, modify,
and distribute it for any purpose, including commercial use, with attribution.

---

## 📎 Citation

If you use this model in your work, please cite:

```
CRNN-CTC Captcha Solver (2026)
Trained on 275k real-world CAPTCHAs from 7 HuggingFace datasets.
Architecture: 6-block CNN + 2× BiLSTM + CTC, 10M parameters, 38 MB.
Sequence accuracy: 90.08% on held-out real test set.
License: MIT
```