KaptchaOCR — OCR, glyph detection and slider matching in one binary

KaptchaOCR models

ONNX recognition and glyph-detection models for KaptchaOCR — a Rust CAPTCHA library and small HTTP server.

KaptchaOCR reads text CAPTCHAs, finds the character boxes used by click CAPTCHAs, and solves slider puzzles. These are the same model files that ship in the GitHub repository. No training code or datasets are included here.

Files

Path	Task	Size	Use
`model/finetuned/tiny_rec.onnx` + `.json`	recognition	4.3 MB	`fast` profile
`model/finetuned/small_rec.onnx` + `.json`	recognition	20 MB	`balanced` profile (default)
`model/finetuned/medium_rec.onnx` + `.json`	recognition	73 MB	`accurate` profile
`model/finetuned/small_det.onnx`	glyph detection	9.5 MB	click CAPTCHAs
`model/finetuned/medium_det.onnx`	glyph detection	59 MB	click CAPTCHAs, tighter boxes
`model/stock_common.onnx` + `.json`	recognition	52 MB	ddddocr v2 fallback
`model/stock_old.onnx` + `.json`	recognition	13 MB	ddddocr legacy fallback
`model/stock_det.onnx`	detection	19 MB	ddddocr detector fallback
`model/models.json`	manifest	—	profiles and model names

Quick start

The models run with the KaptchaOCR binary. Build it from the GitHub repo, then point the server at this model folder:

git clone https://github.com/ozalt/kaptchaocr
cd kaptchaocr
cargo build --release

# download this repo next to it
hf download zaltahir/kaptchaocr --local-dir ../kaptchaocr-models

# run from the folder that contains model/
cd ../kaptchaocr-models
../kaptchaocr/target/release/kaptchaocr \
  --ocr --det --slide \
  --manifest model/models.json --profile balanced

--profile fast uses ft-tiny, balanced uses ft-small, and accurate uses ft-medium. For click CAPTCHAs, add the glyph detector explicitly:

kaptchaocr --ocr --det --slide \
  --manifest model/models.json \
  --rec-model ft-small --det-model ft-small-det

You can also clone this repo with Git LFS:

git clone https://huggingface.co/zaltahir/kaptchaocr

Benchmarks

Recognition on six held-out CAPTCHA sets (4,399 images), exact string match, case-sensitive. Latency is mean ms/image on an Intel i5-1145G7, single process.

Model	Exact	Char	Latency	Size
KaptchaOCR ft-medium	90.7%	96.0%	69 ms	73 MB
KaptchaOCR ft-small	80.4%	93.0%	28 ms	20 MB
KaptchaOCR ft-tiny	67.0%	88.8%	10 ms	4.3 MB
PP-OCRv6 medium (base)	28.6%	59.0%	75 ms	73 MB
ddddocr legacy (stock)	15.3%	42.2%	29 ms	13 MB
ddddocr v2 (stock)	14.3%	42.3%	33 ms	52 MB
PP-OCRv6 tiny (base)	8.9%	38.1%	10 ms	4.3 MB

Glyph detection on a fresh synthetic set (400 images, 1,800 boxes):

Model	F1@0.5	F1@0.75	Latency	Size
KaptchaOCR ft-small	95.8%	50.5%	270 ms	9.5 MB
KaptchaOCR ft-medium	95.7%	63.1%	1,315 ms	59 MB
ddddocr det (stock)	57.5%	32.4%	112 ms	19 MB
PP-OCRv6 medium (base)	32.0%	7.0%	1,305 ms	59 MB

Caveats: the recognition sets are image-disjoint from training but come from the same public sources, so this is not an out-of-distribution benchmark. The detection set is a synthetic proxy; real click-CAPTCHA data is not included. See the full benchmark page for the raw CSVs, charts and method.

How the models work

Recognition models take an RGB image resized to height 48 (dynamic width) and decode a CTC character sequence. The matching charset JSON is required.
Fine-tuned detection models are DB (PaddleOCR-style) glyph detectors that return a probability map, turned into boxes by the server.
Stock detection is the ddddocr YOLO-style detector.
model/models.json wires the paths, charsets and post-processing thresholds.

The models are inference-only. The Rust code handles image preparation, CTC decoding, box post-processing, click ordering and slider matching.

Licenses and attribution

KaptchaOCR code and the fine-tuned models: Apache-2.0.
Stock model/stock_* models: MIT, from ddddocr. The notice is in THIRD_PARTY_LICENSES.md.
Base PP-OCRv6 models and the fine-tuned derivatives: Apache-2.0, from PaddleOCR.

Scope

KaptchaOCR does not solve CAPTCHAs that ask you to pick pictures of cars, trucks, buses, traffic lights, or other objects. That is a different job and is still work in progress. It is not included in this repo yet.

zaltahir
/

kaptchaocr