|
Download README.md from bxding/recaptcha-classifier: direct link, hf CLI and curl.
- Browser
- Download file 6.59 kB
-
https://huggingface.co/bxding/recaptcha-classifier/resolve/main/README.md
- Command line
-
hf download hf://bxding/recaptcha-classifier/README.md
-
curl -L -o README.md https://huggingface.co/bxding/recaptcha-classifier/resolve/main/README.md
6.59 kB
| license: other | |
| license_name: mit-apache-2.0-dinov3 | |
| license_link: https://huggingface.co/bxding/RC1/blob/main/NOTICE.md | |
| library_name: onnx | |
| pipeline_tag: image-classification | |
| base_model: | |
| - timm/mobilenetv4_conv_medium.e250_r384_in12k_ft_in1k | |
| - timm/convnext_tiny.dinov3_lvd1689m | |
| datasets: | |
| - bxding/recaptcha-67k | |
| tags: | |
| - computer-vision | |
| - image-classification | |
| - onnx | |
| # recaptcha-classifier (RC1) | |
| RC1 is a pair of small, fast image recognition models that select the matching | |
| cells in reCAPTCHA v2 image grids. The 3×3 model classifies each of the nine | |
| cells separately, while the 4×4 model processes the whole grid at once. Both | |
| run locally on a CPU with ONNX Runtime. On my $2 VPS, the 3×3 model processes | |
| about **9 grids per second** using two CPU threads. | |
|  | |
| *Inference in this demo ran on my $2 VPS. The left window shows a 4×4 grid and | |
| the right window shows a 3×3 grid.* | |
| ## Models | |
| | Model | Backbone | Input | Supported targets | | |
| | --- | --- | --- | --- | | |
| | `grid9.onnx` | MobileNetV4 (`mobilenetv4_conv_medium.e250_r384_in12k_ft_in1k`) | Nine cells from a 3×3 grid, each resized to 224×224 | bicycle, bus, car, crosswalk, fire hydrant, motorcycle\*, parking meter\*, stairs\*, traffic light\* | | |
| | `grid16.onnx` | ConvNeXt Tiny (`convnext_tiny.dinov3_lvd1689m`) | One 4×4 grid image, resized to 448×448 | bicycle, bus, chimney\*, crosswalk, fire hydrant, motorcycle, parking meter\*, stairs, taxi\*, tractor, traffic light | | |
| `grid16.onnx` is built with DINOv3 and is distributed under the | |
| [DINOv3 License](LICENSE-DINOv3.md), not the MIT License. See [License](#license). | |
| \* These targets are rare. Across all splits, the 3×3 dataset has 14 grids | |
| for motorcycle, 7 for traffic light, 2 for parking meter, and 1 for stairs. | |
| The 4×4 dataset has 7 grids for parking meter, 13 for chimney, and 108 for taxi. | |
| Each model outputs a probability for every cell. The inference script selects | |
| the cells at or above one tuned threshold per model: 0.475 for `grid9.onnx` | |
| and 0.45 for `grid16.onnx`. The threshold is stored in the model file. | |
| ## Evaluation | |
| Accuracy of the released ONNX models on | |
| [recaptcha-67k](https://huggingface.co/datasets/bxding/recaptcha-67k). | |
| *Exact* means every cell prediction matched the reference label. | |
| | Model | Split | Grids | Exact | Cell precision | Cell recall | Cell F1 | | |
| | --- | --- | ---: | ---: | ---: | ---: | ---: | | |
| | `grid9.onnx` | `review` | 300 | 81.0% | 0.978 | 0.948 | 0.963 | | |
| | `grid9.onnx` | `test` (random 600) | 600 | 82.5% | 0.976 | 0.958 | 0.967 | | |
| | `grid16.onnx` | `review` | 282 | 63.5% | 0.915 | 0.939 | 0.927 | | |
| | `grid16.onnx` | `test` (random 600) | 600 | 65.7% | 0.944 | 0.948 | 0.946 | | |
| These numbers measure agreement with model-generated labels, not live | |
| challenge pass rates: | |
| - The `test` labels come from the same labelers RC1 was trained on: Qwen for | |
| 3×3 and SAM 3 for 4×4. | |
| - The `review` labels were checked by Claude Opus 5.5, not by humans. | |
| - The original machine labels exactly matched Claude’s review on 81.3% of | |
| 3×3 grids and 70.6% of 4×4 grids. | |
| Among 4×4 targets with at least 10 reviewed grids, exact agreement was lowest | |
| for stairs (10/18), traffic lights (29/48), and motorcycles (52/86). | |
| ## Running inference | |
| Requires Python 3.11 or newer. From this directory, install the dependencies | |
| and run the prediction script: | |
| ```sh | |
| python -m pip install onnxruntime numpy pillow | |
| python predict_onnx.py --model weights/grid16.onnx --image challenge.jpg --target buses | |
| ``` | |
| Use `weights/grid9.onnx` for 3×3 grids and `weights/grid16.onnx` for 4×4 grids. | |
| The script prints JSON with a probability for each cell and a `selected_cells` | |
| list. Cells are numbered from 0, left to right and then top to bottom. | |
| To limit the number of CPU threads, pass `--threads`. For all options, run | |
| `python predict_onnx.py --help`. | |
| ## Benchmarks | |
| Approximate CPU throughput once the model is loaded. Times include image | |
| loading and preprocessing. | |
| | Machine | CPU threads | 3×3 grids/sec | 4×4 grids/sec | | |
| | --- | ---: | ---: | ---: | | |
| | Apple M2 Pro | 4 | ~10 | ~7 | | |
| | $2 VPS | 2 | ~9 | ~4 | | |
| ## Training and export | |
| Training requires Python 3.11, [uv](https://docs.astral.sh/uv/), and a local | |
| copy of the recaptcha-67k dataset. From this directory: | |
| ```sh | |
| # Install dependencies | |
| uv sync --locked | |
| # Prepare the dataset | |
| uv run python train.py prepare --data /path/to/recaptcha-67k --output runs/rc | |
| # Train the 3×3 and 4×4 models | |
| uv run python train.py train --manifest runs/rc/split.json --output runs/rc --tiles 9 | |
| uv run python train.py train --manifest runs/rc/split.json --output runs/rc --tiles 16 --backbone convnext_tiny.dinov3_lvd1689m | |
| # Export to ONNX | |
| uv run --group export python train.py export --checkpoint runs/rc/grid9/best.pt | |
| # Run tests | |
| uv run --group export python -m unittest test_training | |
| ``` | |
| Notes: | |
| - Training uses a GPU if one is available. On CUDA, add `--amp` to reduce | |
| memory usage. | |
| - The `prepare` step keeps the dataset's existing splits and leaves out the | |
| `excluded` split. | |
| - All training outputs are written locally to `runs/`. | |
| - For dataset and training options, run `python train.py --help` or add | |
| `--help` to any subcommand. | |
| ## Roadmap | |
| - Improve 4×4 accuracy, which trails both the 3×3 model and the SAM 3 labels | |
| it was trained on. | |
| - Improve chimney and parking meter recognition with more training data. | |
| - Add training data and model support for bridges. | |
| ## Disclaimer | |
| RC1 was created for educational and research purposes. It is not affiliated | |
| with or endorsed by Google or reCAPTCHA. Users are responsible for complying | |
| with all applicable laws and terms of service. | |
| ## License | |
| RC1's files are under three licenses. [NOTICE.md](NOTICE.md) has the details. | |
| | Files | License | | |
| | --- | --- | | |
| | Source code and documentation | [MIT](LICENSE) | | |
| | `weights/grid9.onnx` | [MIT](LICENSE), plus [Apache-2.0](LICENSE-APACHE-2.0.txt) for its pretrained [MobileNetV4 backbone](https://huggingface.co/timm/mobilenetv4_conv_medium.e250_r384_in12k_ft_in1k) | | |
| | `weights/grid16.onnx` | [DINOv3 License](LICENSE-DINOv3.md) only | | |
| `grid16.onnx` is fine-tuned from Meta's | |
| [DINOv3 ConvNeXt Tiny](https://huggingface.co/timm/convnext_tiny.dinov3_lvd1689m) | |
| (built with DINOv3). The DINOv3 License requires derivatives to be distributed | |
| under its own terms, so the MIT License does not apply to this file. Anyone who | |
| uses or redistributes it must follow the DINOv3 License, including its trade | |
| control and use restrictions. | |
| Third-party imagery in the training dataset and demo is outside the scope of | |
| these licenses. | |