🔐 CRNN-CTC Captcha Solver

Convolutional Recurrent Neural Network · Real-Data Fine-Tuned

Alphanumeric CAPTCHA recognition using a deep CRNN + CTC architecture trained on 275,000 real-world labeled CAPTCHAs from 7 HuggingFace datasets, then refined over a full weekend of augmented training on an NVIDIA A100. Achieves 90.08% whole-CAPTCHA sequence accuracy on the held-out real test set.

📋 Model Details

Property	Value
Task	Alphanumeric CAPTCHA Recognition (OCR)
Architecture	CRNN — 6-block CNN + 2× Bidirectional LSTM + Dense
Input	RGB image `(64 × 200 × 3)`, float32 `[0, 1]`
Output	Character sequence, length 1–8
Vocabulary	`0-9`, `a-z`, `A-Z` — 62 characters + CTC blank
Loss	Connectionist Temporal Classification (CTC)
Parameters	10,049,535 (~38.3 MB)
Framework	TensorFlow 2.21 / Keras 3
Training hardware	NVIDIA A100 80 GB

📊 Performance

Overall (held-out real test set, n = 5,502)

Metric	Score
Sequence accuracy (whole CAPTCHA correct)	90.08 %
Character accuracy	96.14 %
CTC loss	1.10

Per-dataset breakdown

Dataset	n (test)	Char acc	Seq acc
cybertruck32489/captcha_90k	1,224	99.9 %	99.7 %
ayoubkirouane/captcha	192	99.9 %	99.5 %
yuxi5/text-captcha-data-clean	1,171	93.6 %	90.9 %
AvinashRicky/CaptchaOCR-500K	1,215	97.8 %	88.7 %
lumasik/captcha-25k	502	96.0 %	86.5 %
yusuf802/captcha_dataset	1,196	92.5 %	80.8 %
ThangaTharun/captchaimages	2	100.0 %	100.0 %

Total params: 10,049,535 — Model size: 38.3 MB

📦 Training Datasets

Dataset	Label field	Images used	Notes
ayoubkirouane/captcha	`solution`	10,000	clean, parquet
ThangaTharun/captchaimages	`output`	100	small, noisy
yuxi5/text-captcha-data-clean	`label`	60,000	capped at 60k
AvinashRicky/CaptchaOCR-500K	`text`	60,000	capped at 60k
cybertruck32489/captcha_90k	`solve`	60,000	capped at 60k
yusuf802/captcha_dataset	`label`	60,000	capped at 60k
lumasik/captcha-25k	`label`	25,000	all images used
Synthetic (17 generator types)	generated	mixed in at 30%	custom Python generators

Total real images preprocessed: 275,100 → 264,096 train / 5,502 val / 5,502 test

All images were resized to 64 × 200 with aspect-preserving letterboxing (pad with median border color).

⚙️ Training Details

Two-stage process

Stage 1 — Synthetic pre-training Training from scratch on 17 procedural CAPTCHA types (custom Python generators) covering different fonts, noise levels, distortions, and color palettes.

Stage 2 — Real-data fine-tuning + augmented weekend run

Initialized from the synthetic checkpoint.
Fine-tuned on 264k real images for 40 epochs (LR 1e-3 → 1e-5 cosine).
Extended with a full weekend run (386 epochs, ~60h on A100) combining:
- 30% synthetic mixed into each batch for robustness.
- GPU augmentation: RandomRotation(±2°), RandomTranslation, RandomZoom(6%), RandomContrast(25%), brightness jitter ±0.08, Gaussian noise σ=0.03.
- Weak-source oversampling: datasets below 90% test seq were oversampled up to ×2 proportionally.

Hyperparameters

Parameter	Value
Optimizer	Adam
Peak LR	5 × 10⁻⁴
Min LR	1 × 10⁻⁶
LR schedule	Warmup + Cosine Decay
Batch size	128
Dropout	0.25
Gradient clip norm	5.0
Max sequence length	8 characters

🚀 Usage

Install dependencies

pip install tensorflow pillow numpy

Predict from an image

import numpy as np
from PIL import Image
import tensorflow as tf

# ── helpers ──────────────────────────────────────────────────────────────────
CHARS       = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"
IDX_TO_CHAR = {i: c for i, c in enumerate(CHARS)}
BLANK_IDX   = 62
IMG_H, IMG_W = 64, 200

def preprocess(pil_img: Image.Image) -> np.ndarray:
    """Aspect-preserving resize to 64×200, pad with median border color."""
    img = pil_img.convert("RGB")
    w, h = img.size
    new_w = int(round(w * IMG_H / h))
    img = img.resize((new_w, IMG_H), Image.LANCZOS)
    if new_w >= IMG_W:
        img = img.resize((IMG_W, IMG_H), Image.LANCZOS)
        return np.array(img, dtype=np.float32) / 255.0
    canvas = np.array(img, dtype=np.float32)
    border_color = np.median(
        np.concatenate([canvas[0], canvas[-1],
                        canvas[:, 0], canvas[:, -1]], axis=0), axis=0
    )
    pad_l = (IMG_W - new_w) // 2
    result = np.full((IMG_H, IMG_W, 3), border_color, dtype=np.float32)
    result[:, pad_l:pad_l + new_w] = canvas
    return result / 255.0

def decode_ctc(logits: np.ndarray) -> str:
    """Greedy CTC decode — collapse repeats, remove blanks."""
    indices = np.argmax(logits, axis=-1)       # (T,)
    chars, prev = [], -1
    for idx in indices:
        if idx != prev and idx != BLANK_IDX:
            chars.append(IDX_TO_CHAR.get(int(idx), ""))
        prev = idx
    return "".join(chars)

# ── load model ───────────────────────────────────────────────────────────────
# Clone the repo or download the weights file, then:
from model import build_crnn_model   # from this repository

model = build_crnn_model()
model.load_weights("weekend_best.weights.h5")

# ── run inference ─────────────────────────────────────────────────────────────
img    = Image.open("captcha.png")
tensor = preprocess(img)[np.newaxis]           # (1, 64, 200, 3)
logits = model(tensor, training=False).numpy() # (1, 50, 63)
print("Prediction:", decode_ctc(logits[0]))

Batch inference

images = [preprocess(Image.open(p)) for p in image_paths]
batch  = np.stack(images)                      # (N, 64, 200, 3)
logits = model(batch, training=False).numpy()  # (N, 50, 63)
preds  = [decode_ctc(l) for l in logits]

🖼️ Preprocessing

Input images go through aspect-preserving letterboxing:

Resize height to 64 px, keeping aspect ratio.
If the resulting width ≥ 200 px, squeeze to 200 px.
Otherwise, pad left and right with the median border color of the image to reach 200 px.
Normalize to float32 [0, 1].

This ensures the character shapes are never stretched, which is critical for distinguishing similar characters (e.g. O / 0, l / 1).

⚖️ License

This model is released under the MIT License. You are free to use, copy, modify, and distribute it for any purpose, including commercial use, with attribution.

📎 Citation

If you use this model in your work, please cite:

CRNN-CTC Captcha Solver (2026)
Trained on 275k real-world CAPTCHAs from 7 HuggingFace datasets.
Architecture: 6-block CNN + 2× BiLSTM + CTC, 10M parameters, 38 MB.
Sequence accuracy: 90.08% on held-out real test set.
License: MIT

Downloads last month: -

Wall3
/

Yara_Captcha_Solver