| # Evaluation |
|
|
| This document defines an **offline** evaluation protocol for the PHPWind |
| captcha OCR model. It measures image-recognition quality on labelled captcha |
| images; it does not describe or endorse automated login attempts. |
|
|
| ## Scope |
|
|
| Evaluate only images from PHPWind deployments you own or are explicitly |
| authorized to test. Keep the evaluation dataset separate from the training |
| dataset, and do not publish captcha images or labels that contain private data. |
|
|
| ## Version boundary |
|
|
| The published checkpoint was trained on four-digit numeric captcha images from |
| one target deployment whose footer displayed `v0.7β`. This footer text is an |
| observed deployment label, not an official PHPWind release version claim. |
|
|
| | Deployment or version label | Status | Evidence | Notes | |
| |---|---|---|---| |
| | Target deployment — footer label `v0.7β` | Training and reference-evaluation scope | 997 manually labelled images; 88.61% held-out validation accuracy | The only visual configuration represented in the reported data. | |
| | Other PHPWind releases, forks, themes, or captcha generators | Unverified | No version-specific evaluation | Evaluate with authorized representative samples and fine-tune when the image distribution differs. | |
|
|
| ## Reference metric |
|
|
| The published checkpoint reports the following held-out validation result: |
|
|
| | Metric | Value | |
| |---|---:| |
| | Validation accuracy | 88.61% | |
| | Labelled image count | 997 | |
| | Validation split | 8% | |
| | Task | Four-digit numeric captcha recognition for the target `v0.7β` footer-label deployment | |
|
|
| This metric is a historical reference for this checkpoint. It is not a |
| production guarantee: other PHPWind versions, themes, fonts, and generator |
| settings can change the image distribution. |
|
|
| ## Reproducible protocol |
|
|
| 1. Collect labelled PHPWind captcha images only from an authorized deployment. |
| 2. Keep a fixed held-out split that is not used for training or tuning. |
| 3. Apply the model's preprocessing exactly: RGB, resize to `160 × 64` with |
| bilinear interpolation, then divide pixel values by `255`. |
| 4. Decode each of the four output positions by argmax and compare the complete |
| predicted code with its label. |
| 5. Report exact-match accuracy together with dataset size, split method, model |
| revision, and PHPWind version/theme details needed to interpret the result. |
|
|
| ## Recommended report format |
|
|
| | Field | What to record | |
| |---|---| |
| | Model revision | Commit hash or file checksum | |
| | PHPWind environment | Version and relevant theme/customization details | |
| | Dataset provenance | Authorized source and collection date range | |
| | Split | Fixed seed or explicit file list | |
| | Metric | Complete four-digit exact-match accuracy | |
| | Limitations | Distribution differences, labelling issues, and known failures | |
|
|
| For adapting the checkpoint to a changed PHPWind generator, see the |
| [training guide](TRAINING.md). |
|
|
