README.md

metadata

license: apache-2.0
language:
  - en
  - ru
library_name: transformers
pipeline_tag: image-to-text
tags:
  - captcha
  - ocr
  - minecraft
  - pp-ocrv6
  - text-recognition
  - cyrillic
datasets:
  - Titlehhhh/captchaocr-dataset
base_model:
  - PaddlePaddle/PP-OCRv6_medium_rec_safetensors

CaptchaOCR PP-OCRv6 — captcha text recogniser

A PP-OCRv6 fine-tune that reads a captcha image, usually the answer band: 18 evaluated styles, Latin letters, digits, and part of the Cyrillic alphabet.

Use it in 10 lines

import torch
from PIL import Image
from transformers import PPOCRV6SmallRecForTextRecognition
from transformers import PPOCRV6SmallRecImageProcessor

REPO = "Titlehhhh/captchaocr-ppocrv6"
model = PPOCRV6SmallRecForTextRecognition.from_pretrained(REPO).eval()
proc = PPOCRV6SmallRecImageProcessor.from_pretrained(REPO)
inputs = proc(images=[Image.open("band.png").convert("RGB")], return_tensors="pt")
with torch.no_grad():
    logits = model(**inputs)
print(proc.post_process_text_recognition(logits))  # [{'text': 'ly8r', 'score': 0.96}]

band.png is a tight crop of the answer band, not a screenshot. The processor resizes it to 48 × 320, normalises it, and decodes CTC with the 18 762-entry dictionary from preprocessor_config.json. This exact snippet was run against these weights.

Manual CTC decoding

If you would rather decode by hand, the forward returns the logits in last_hidden_state:

with open("character_list.txt", encoding="utf-8") as f:
    dictionary = [line.rstrip("\r\n") for line in f]   # rstrip, not strip: " " is a class
ids = logits.last_hidden_state.argmax(-1)[0].tolist()
text, prev = [], -1
for i in ids:
    if i != prev and i != 0:        # collapse repeats, drop the blank
        text.append(dictionary[i])
    prev = i
print("".join(text))

What it supports

Supports	Does not support
18 evaluated styles: 11 from the published dataset snapshot, 1 unpublished (LimboFilter/digits), 6 held-out plugins	3 types that draw a number as a word while the answer is digits: ru-numberword, ru-numberword-caption, LimboFilter/map. Those need a solver step, not OCR
Latin letters, digits, and 52 of 66 Cyrillic letters: all 33 uppercase, 19 lowercase	14 lowercase Cyrillic letters are not classes: б, г, ё, ж, з, й, л, у, ф, х, щ, ъ, э, ю. Characters outside character_list.txt are never produced
A tight crop of the answer band (roi in the dataset), noisy bands with lines, per-character shifts, coloured backgrounds	A whole game screenshot without a cropper: 2 of 6 screenshots read correctly as-is, 6 of 6 with the project's cropper
Styles close to the trained plugins, including other plugins of the same visual family	A style absent from training: expect 0.57–0.98 exact, never 1.0

Results

Exact match and CER after greedy CTC decoding, on 11 400 images of 18 styles: 200 rows per trained style plus 1 500 rows per held-out plugin. The median is taken over the 18 per-style scores; the micro average counts every image, and 9 000 of the 11 400 belong to held-out plugins.

Metric	Score
median exact match (18 styles)	0.8875
median CER	0.0344
micro exact match (11 400 images)	0.7797
micro CER	0.0755

Plugins never seen in training — what to expect on a new style:

Plugin	Style	Exact	CER
CaptchaVerifyZ	map	0.9780	0.0045
Minecaptcha	map	0.9373	0.0162
BotFilter	map	0.8140	0.0508
sonar	legacy	0.7693	0.0793
sonar	complex	0.6253	0.1293
sonar	standard	0.5707	0.1671

Per-style results on the trained styles

Sample-level hold-out: the first 200 rows of each style were taken out of training.

Style	Exact	CER
Archy-X/Krypton/map	1.0000	0.0000
ringoame196-s-mcPlugin/ReproductionCAPTCHA/map	1.0000	0.0000
irootdev/AnBot/captcha	0.9950	0.0017
haq/mapcha/map	0.9550	0.0112
GalleryCaptcha/ru-web	0.9250	0.0150
just-captcha/ProstoCaptcha/map	0.9150	0.0150
LimboFilter/digits (unpublished)	0.8900	0.0288
GalleryCaptcha/ru-symbols	0.8850	0.0400
LimboFilter/symbols	0.8000	0.0783
LagBug/XProtect/code	0.3600	0.1542
LagBug/XProtect/map	0.2700	0.1850
InstantlyMoist/Captcha/textcaptcha	0.2400	0.4220

Limitations

• Four of the twelve trained styles are fed the whole frame, not a band. Three of them are the weakest: XProtect/code 0.36, XProtect/map 0.27, textcaptcha 0.24. The fourth, AnBot/captcha, scores 0.9950 on the frame — the input is not what makes a style hard.
• The median is carried by the easy styles. Seven of the twelve trained styles score 0.89 or better. On a style unlike the training set, read the held-out table instead: 0.57–0.98.
• Screenshots need a cropper. Feed a raw game frame and 2 of 6 real screenshots are read correctly; with the cropper, 6 of 6. The cropper ships with the code, not with these weights.
• The micro average is mostly held-out data (9 000 of 11 400 images), so it answers "how does it do on unseen plugins", not "accuracy in general".
• It is not an API and not a solver for new plugins. One image in, one string out; a style absent from training has no guarantee.

Dataset

Titlehhhh/captchaocr-dataset is synthetic: every image is a re-render by this project's Java generators of a captcha drawn by an open-source Minecraft plugin. The published snapshot holds 20 styles and 200 000 samples. A 21st style, out_ru/LimboFilter/digits, was generated after the export and is not in the snapshot — which is why 18 evaluated styles = 11 from the snapshot + 1 unpublished + 6 held out.

Training

• Base: PP-OCRv6_medium_rec (Apache-2.0). The CTC head was kept and grown from 18 710 to 18 762 classes: rows 18 710–18 761 are 52 appended Cyrillic characters.
• Initialised from an earlier Cyrillic run of the same recipe that replayed Latin data, then fine-tuned on 12 styles.
• 5 epochs × 488 steps = 2 440 steps, batch 32, learning rate 8e-6 for backbone and head, weight decay 0.01, gradient clipping 1.0.
• 15 594 training examples: 1 300 per style after a 200-row hold-out, from a 10 000-per-style render (the run used --limit-per-type 1500).
• 33.9 min on one GTX 1080 Ti, 108 images/s, fp16.
• Whole-frame styles were trained with the frame as input; the rest with the roi band.
• Full command line and scripts: github.com/Titlehhhh/CaptchaOCR.

License & citation

The weights are Apache-2.0, as a derivative of PP-OCRv6_medium_rec. The training data is published under mixed per-plugin licences — some rows are non-commercial, so read the dataset LICENSE before commercial use.

Version history

• universal_v1 (2026-10-27) — current weights, 18 762 classes, no weight interpolation (α = 1.0). It replaced all20, which is no longer downloadable.
• Interpolation is off on purpose: pulling back towards the base damages the appended rows (LimboFilter/symbols 0.765 → 0.016). Replay of old data, not a weight soup, is what keeps them.