---
license: apache-2.0
language:
- en
- ru
library_name: transformers
pipeline_tag: image-to-text
tags:
- captcha
- ocr
- minecraft
- pp-ocrv6
- text-recognition
- cyrillic
datasets:
- Titlehhhh/captchaocr-dataset
base_model:
- PaddlePaddle/PP-OCRv6_medium_rec_safetensors
---
       
# CaptchaOCR PP-OCRv6 — captcha text recogniser
A PP-OCRv6 fine-tune that reads a captcha image, usually the answer band: 18 evaluated styles,
Latin letters, digits, and part of the Cyrillic alphabet.
## Use it in 10 lines
```python
import torch
from PIL import Image
from transformers import PPOCRV6SmallRecForTextRecognition
from transformers import PPOCRV6SmallRecImageProcessor
REPO = "Titlehhhh/captchaocr-ppocrv6"
model = PPOCRV6SmallRecForTextRecognition.from_pretrained(REPO).eval()
proc = PPOCRV6SmallRecImageProcessor.from_pretrained(REPO)
inputs = proc(images=[Image.open("band.png").convert("RGB")], return_tensors="pt")
with torch.no_grad():
logits = model(**inputs)
print(proc.post_process_text_recognition(logits)) # [{'text': 'ly8r', 'score': 0.96}]
```
`band.png` is a tight crop of the answer band, not a screenshot. The processor resizes it to
48 × 320, normalises it, and decodes CTC with the 18 762-entry dictionary from
`preprocessor_config.json`. This exact snippet was run against these weights.
Manual CTC decoding
If you would rather decode by hand, the forward returns the logits in `last_hidden_state`:
```python
with open("character_list.txt", encoding="utf-8") as f:
dictionary = [line.rstrip("\r\n") for line in f] # rstrip, not strip: " " is a class
ids = logits.last_hidden_state.argmax(-1)[0].tolist()
text, prev = [], -1
for i in ids:
if i != prev and i != 0: # collapse repeats, drop the blank
text.append(dictionary[i])
prev = i
print("".join(text))
```
## What it supports
| Supports | Does not support |
|---|---|
| **18 evaluated styles**: 11 from the published dataset snapshot, 1 unpublished (`LimboFilter/digits`), 6 held-out plugins | **3 types that draw a number as a word** while the answer is digits: `ru-numberword`, `ru-numberword-caption`, `LimboFilter/map`. Those need a solver step, not OCR |
| Latin letters, digits, and 52 of 66 Cyrillic letters: all 33 uppercase, 19 lowercase | 14 lowercase Cyrillic letters are not classes: б, г, ё, ж, з, й, л, у, ф, х, щ, ъ, э, ю. Characters outside `character_list.txt` are never produced |
| A tight crop of the answer band (`roi` in the dataset), noisy bands with lines, per-character shifts, coloured backgrounds | A whole game screenshot without a cropper: 2 of 6 screenshots read correctly as-is, 6 of 6 with the project's cropper |
| Styles close to the trained plugins, including other plugins of the same visual family | A style absent from training: expect 0.57–0.98 exact, never 1.0 |
## Results
Exact match and CER after greedy CTC decoding, on 11 400 images of 18 styles: 200 rows per trained
style plus 1 500 rows per held-out plugin. The median is taken over the 18 per-style scores; the
micro average counts every image, and 9 000 of the 11 400 belong to held-out plugins.
| Metric | Score |
|---|---:|
| median exact match (18 styles) | **0.8875** |
| median CER | **0.0344** |
| micro exact match (11 400 images) | **0.7797** |
| micro CER | **0.0755** |
**Plugins never seen in training** — what to expect on a new style:
| Plugin | Style | Exact | CER |
|---|---|---:|---:|
| CaptchaVerifyZ | map | 0.9780 | 0.0045 |
| Minecaptcha | map | 0.9373 | 0.0162 |
| BotFilter | map | 0.8140 | 0.0508 |
| sonar | legacy | 0.7693 | 0.0793 |
| sonar | complex | 0.6253 | 0.1293 |
| sonar | standard | 0.5707 | 0.1671 |
Per-style results on the trained styles
Sample-level hold-out: the first 200 rows of each style were taken out of training.
| Style | Exact | CER |
|---|---:|---:|
| `Archy-X/Krypton/map` | 1.0000 | 0.0000 |
| `ringoame196-s-mcPlugin/ReproductionCAPTCHA/map` | 1.0000 | 0.0000 |
| `irootdev/AnBot/captcha` | 0.9950 | 0.0017 |
| `haq/mapcha/map` | 0.9550 | 0.0112 |
| `GalleryCaptcha/ru-web` | 0.9250 | 0.0150 |
| `just-captcha/ProstoCaptcha/map` | 0.9150 | 0.0150 |
| `LimboFilter/digits` (unpublished) | 0.8900 | 0.0288 |
| `GalleryCaptcha/ru-symbols` | 0.8850 | 0.0400 |
| `LimboFilter/symbols` | 0.8000 | 0.0783 |
| `LagBug/XProtect/code` | 0.3600 | 0.1542 |
| `LagBug/XProtect/map` | 0.2700 | 0.1850 |
| `InstantlyMoist/Captcha/textcaptcha` | 0.2400 | 0.4220 |
## Limitations
* **Four of the twelve trained styles are fed the whole frame**, not a band. Three of them are the
weakest: `XProtect/code` 0.36, `XProtect/map` 0.27, `textcaptcha` 0.24. The fourth,
`AnBot/captcha`, scores 0.9950 on the frame — the input is not what makes a style hard.
* **The median is carried by the easy styles.** Seven of the twelve trained styles score 0.89 or
better. On a style unlike the training set, read the held-out table instead: 0.57–0.98.
* **Screenshots need a cropper.** Feed a raw game frame and 2 of 6 real screenshots are read
correctly; with the cropper, 6 of 6. The cropper ships with the code, not with these weights.
* **The micro average is mostly held-out data** (9 000 of 11 400 images), so it answers "how does
it do on unseen plugins", not "accuracy in general".
* **It is not an API and not a solver for new plugins.** One image in, one string out; a style
absent from training has no guarantee.
## Dataset
[`Titlehhhh/captchaocr-dataset`](https://huggingface.co/datasets/Titlehhhh/captchaocr-dataset) is
synthetic: every image is a re-render by this project's Java generators of a captcha drawn by an
open-source Minecraft plugin. The published snapshot holds 20 styles and 200 000 samples. A 21st
style, `out_ru/LimboFilter/digits`, was generated after the export and is not in the snapshot —
which is why 18 evaluated styles = 11 from the snapshot + 1 unpublished + 6 held out.
Training
* Base: `PP-OCRv6_medium_rec` (Apache-2.0). The CTC head was kept and grown from 18 710 to
18 762 classes: rows 18 710–18 761 are 52 appended Cyrillic characters.
* Initialised from an earlier Cyrillic run of the same recipe that replayed Latin data, then
fine-tuned on 12 styles.
* 5 epochs × 488 steps = 2 440 steps, batch 32, learning rate 8e-6 for backbone and head,
weight decay 0.01, gradient clipping 1.0.
* 15 594 training examples: 1 300 per style after a 200-row hold-out, from a 10 000-per-style
render (the run used `--limit-per-type 1500`).
* 33.9 min on one GTX 1080 Ti, 108 images/s, fp16.
* Whole-frame styles were trained with the frame as input; the rest with the `roi` band.
* Full command line and scripts:
[github.com/Titlehhhh/CaptchaOCR](https://github.com/Titlehhhh/CaptchaOCR).
## License & citation
The weights are **Apache-2.0**, as a derivative of `PP-OCRv6_medium_rec`. The training data is
published under **mixed per-plugin licences** — some rows are non-commercial, so read the dataset
[LICENSE](https://huggingface.co/datasets/Titlehhhh/captchaocr-dataset/blob/main/LICENSE) before
commercial use.
Version history
* `universal_v1` (2026-10-27) — current weights, 18 762 classes, no weight interpolation (α = 1.0).
It replaced `all20`, which is no longer downloadable.
* Interpolation is off on purpose: pulling back towards the base damages the appended rows
(`LimboFilter/symbols` 0.765 → 0.016). Replay of old data, not a weight soup, is what keeps them.