recaptcha-classifier (RC1)

RC1 is a pair of small, fast image recognition models that select the matching cells in reCAPTCHA v2 image grids. The 3×3 model classifies each of the nine cells separately, while the 4×4 model processes the whole grid at once. Both run locally on a CPU with ONNX Runtime. On my $2 VPS, the 3×3 model processes about 9 grids per second using two CPU threads.

Example predictions

Inference in this demo ran on my $2 VPS. The left window shows a 4×4 grid and the right window shows a 3×3 grid.

Models

Model Backbone Input Supported targets
grid9.onnx MobileNetV4 (mobilenetv4_conv_medium.e250_r384_in12k_ft_in1k) Nine cells from a 3×3 grid, each resized to 224×224 bicycle, bus, car, crosswalk, fire hydrant, motorcycle*, parking meter*, stairs*, traffic light*
grid16.onnx ConvNeXt Tiny (convnext_tiny.dinov3_lvd1689m) One 4×4 grid image, resized to 448×448 bicycle, bus, chimney*, crosswalk, fire hydrant, motorcycle, parking meter*, stairs, taxi*, tractor, traffic light

grid16.onnx is built with DINOv3 and is distributed under the DINOv3 License, not the MIT License. See License.

* These targets are rare. Across all splits, the 3×3 dataset has 14 grids for motorcycle, 7 for traffic light, 2 for parking meter, and 1 for stairs. The 4×4 dataset has 7 grids for parking meter, 13 for chimney, and 108 for taxi.

Each model outputs a probability for every cell. The inference script selects the cells at or above one tuned threshold per model: 0.475 for grid9.onnx and 0.45 for grid16.onnx. The threshold is stored in the model file.

Evaluation

Accuracy of the released ONNX models on recaptcha-67k. Exact means every cell prediction matched the reference label.

Model Split Grids Exact Cell precision Cell recall Cell F1
grid9.onnx review 300 81.0% 0.978 0.948 0.963
grid9.onnx test (random 600) 600 82.5% 0.976 0.958 0.967
grid16.onnx review 282 63.5% 0.915 0.939 0.927
grid16.onnx test (random 600) 600 65.7% 0.944 0.948 0.946

These numbers measure agreement with model-generated labels, not live challenge pass rates:

  • The test labels come from the same labelers RC1 was trained on: Qwen for 3×3 and SAM 3 for 4×4.
  • The review labels were checked by Claude Opus 5.5, not by humans.
  • The original machine labels exactly matched Claude’s review on 81.3% of 3×3 grids and 70.6% of 4×4 grids.

Among 4×4 targets with at least 10 reviewed grids, exact agreement was lowest for stairs (10/18), traffic lights (29/48), and motorcycles (52/86).

Running inference

Requires Python 3.11 or newer. From this directory, install the dependencies and run the prediction script:

python -m pip install onnxruntime numpy pillow
python predict_onnx.py --model weights/grid16.onnx --image challenge.jpg --target buses

Use weights/grid9.onnx for 3×3 grids and weights/grid16.onnx for 4×4 grids.

The script prints JSON with a probability for each cell and a selected_cells list. Cells are numbered from 0, left to right and then top to bottom.

To limit the number of CPU threads, pass --threads. For all options, run python predict_onnx.py --help.

Benchmarks

Approximate CPU throughput once the model is loaded. Times include image loading and preprocessing.

Machine CPU threads 3×3 grids/sec 4×4 grids/sec
Apple M2 Pro 4 ~10 ~7
$2 VPS 2 ~9 ~4

Training and export

Training requires Python 3.11, uv, and a local copy of the recaptcha-67k dataset. From this directory:

# Install dependencies
uv sync --locked

# Prepare the dataset
uv run python train.py prepare --data /path/to/recaptcha-67k --output runs/rc

# Train the 3×3 and 4×4 models
uv run python train.py train --manifest runs/rc/split.json --output runs/rc --tiles 9
uv run python train.py train --manifest runs/rc/split.json --output runs/rc --tiles 16 --backbone convnext_tiny.dinov3_lvd1689m

# Export to ONNX
uv run --group export python train.py export --checkpoint runs/rc/grid9/best.pt

# Run tests
uv run --group export python -m unittest test_training

Notes:

  • Training uses a GPU if one is available. On CUDA, add --amp to reduce memory usage.
  • The prepare step keeps the dataset's existing splits and leaves out the excluded split.
  • All training outputs are written locally to runs/.
  • For dataset and training options, run python train.py --help or add --help to any subcommand.

Roadmap

  • Improve 4×4 accuracy, which trails both the 3×3 model and the SAM 3 labels it was trained on.
  • Improve chimney and parking meter recognition with more training data.
  • Add training data and model support for bridges.

Disclaimer

RC1 was created for educational and research purposes. It is not affiliated with or endorsed by Google or reCAPTCHA. Users are responsible for complying with all applicable laws and terms of service.

License

RC1's files are under three licenses. NOTICE.md has the details.

Files License
Source code and documentation MIT
weights/grid9.onnx MIT, plus Apache-2.0 for its pretrained MobileNetV4 backbone
weights/grid16.onnx DINOv3 License only

grid16.onnx is fine-tuned from Meta's DINOv3 ConvNeXt Tiny (built with DINOv3). The DINOv3 License requires derivatives to be distributed under its own terms, so the MIT License does not apply to this file. Anyone who uses or redistributes it must follow the DINOv3 License, including its trade control and use restrictions.

Third-party imagery in the training dataset and demo is outside the scope of these licenses.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for bxding/recaptcha-classifier

Quantized
(1)
this model

Dataset used to train bxding/recaptcha-classifier