Evaluation
This document defines an offline evaluation protocol for the PHPWind captcha OCR model. It measures image-recognition quality on labelled captcha images; it does not describe or endorse automated login attempts.
Scope
Evaluate only images from PHPWind deployments you own or are explicitly authorized to test. Keep the evaluation dataset separate from the training dataset, and do not publish captcha images or labels that contain private data.
Version boundary
The published checkpoint was trained on four-digit numeric captcha images from
one target deployment whose footer displayed v0.7β. This footer text is an
observed deployment label, not an official PHPWind release version claim.
| Deployment or version label | Status | Evidence | Notes |
|---|---|---|---|
Target deployment — footer label v0.7β |
Training and reference-evaluation scope | 997 manually labelled images; 88.61% held-out validation accuracy | The only visual configuration represented in the reported data. |
| Other PHPWind releases, forks, themes, or captcha generators | Unverified | No version-specific evaluation | Evaluate with authorized representative samples and fine-tune when the image distribution differs. |
Reference metric
The published checkpoint reports the following held-out validation result:
| Metric | Value |
|---|---|
| Validation accuracy | 88.61% |
| Labelled image count | 997 |
| Validation split | 8% |
| Task | Four-digit numeric captcha recognition for the target v0.7β footer-label deployment |
This metric is a historical reference for this checkpoint. It is not a production guarantee: other PHPWind versions, themes, fonts, and generator settings can change the image distribution.
Reproducible protocol
- Collect labelled PHPWind captcha images only from an authorized deployment.
- Keep a fixed held-out split that is not used for training or tuning.
- Apply the model's preprocessing exactly: RGB, resize to
160 × 64with bilinear interpolation, then divide pixel values by255. - Decode each of the four output positions by argmax and compare the complete predicted code with its label.
- Report exact-match accuracy together with dataset size, split method, model revision, and PHPWind version/theme details needed to interpret the result.
Recommended report format
| Field | What to record |
|---|---|
| Model revision | Commit hash or file checksum |
| PHPWind environment | Version and relevant theme/customization details |
| Dataset provenance | Authorized source and collection date range |
| Split | Fixed seed or explicit file list |
| Metric | Complete four-digit exact-match accuracy |
| Limitations | Distribution differences, labelling issues, and known failures |
For adapting the checkpoint to a changed PHPWind generator, see the training guide.