Card: add the other-captcha-models table (12 foreign models, one protocol) and our own models (a different protocol)

README.md

@@ -138,6 +138,46 @@ same repository.
 | exact match (micro) on the same 29 types, whole frame | 0.0345 | `reports/bench_models/ppocrv6-medium.json` |
 | exact match (micro) on all 46 types, band crop | 0.4043 | `reports/bench_models/ppocrv6-medium.json` |
 
+### Other captcha models measured on this dataset
+
+All foreign models were measured by **one protocol** (`bench/bench_models.py`): the picture is
+cropped by the answer band (`roi`), the input is normalised the same way for everybody, the greedy
+decode is the same, and the metric is exact match on the normalised `text`. The set is the
+**29 band-carrying types, 20 images per type (580 items)**, on the earlier 100-images-per-type
+render of the dataset.
+
+| model | band exact match | item set | source file |
+|---|---|---|---|
+| `PaddlePaddle/PP-OCRv6_medium_rec_onnx` | 0.5517 | 29 types / 580 items | `reports/bench_models/ppocrv6-medium.json` |
+| `PaddlePaddle/PP-OCRv5_mobile_rec_onnx` | 0.4621 | 29 types / 580 items | `reports/bench_models/ppocrv5-mobile.json` |
+| `PaddlePaddle/cyrillic_PP-OCRv5_mobile_rec_onnx` | 0.4621 | 29 types / 580 items | `reports/bench_models/ppocrv5-cyrillic.json` |
+| `easyocr` | 0.3281 | 8 types / 64 items (the old `legacy64` subset) | `reports/bench_models/easyocr.json` |
+| `anuashok/ocr-captcha-v3` | 0.2966 | 29 types / 580 items | `reports/bench_models/anuashok-v3.json` |
+| `graycat660/doctr-crnn-vgg16-bn-russian-onnx` | 0.2776 | 29 types / 580 items | `reports/bench_models/doctr-crnn-ru.json` |
+| `Graf-J/captcha-crnn-base` | 0.0828 | 29 types / 580 items | `reports/bench_models/grafj-crnn-base.json` |
+| `Graf-J/captcha-crnn-finetuned` | 0.0672 | 29 types / 580 items | `reports/bench_models/grafj-crnn.json` |
+| `Graf-J/captcha-conv-transformer-finetuned` | 0.0638 | 29 types / 580 items | `reports/bench_models/grafj-convtr.json` |
+| `emelnov/ocr-captcha-v4-mailru` | 0.0431 | 29 types / 580 items | `reports/bench_models/emelnov-mailru.json` |
+| `jtrecenti/txtcaptcha-crnn` | 0.0034 | 29 types / 580 items | `reports/bench_models/txtcaptcha-crnn.json` |
+| `microsoft/trocr-base-printed` | **no score — the harness failed**: `ValueError: You need to specify either 'text' or 'text_target'.` | — | `reports/bench_models/trocr-base-printed.json` |
+
+For the same base model, the whole frame instead of the band gives **0.0345** — 16× worse. The
+cropper, not the model, decides this task.
+
+### Our own models — **a different protocol, not comparable with the table above**
+
+| our model | median exact (held-out types) | micro exact | held-out set | source |
+|---|---|---|---|---|
+| **this repository: PP-OCRv6 + interpolation α = 0.4** | **0.638** | **0.6123** | 7 types / 3 500 images | `reports/TRAINING_PPOCR_V6_BIG.md` |
+| our CRNN+CTC, trained from scratch | 0.372 | 0.381 | 4 types / 1 999 images | `reports/TRAINING_CRNN_BIG.md` |
+| our CRNN+CTC on its **own training plugins** | 1.000 | 1.000 | 7 types — pure memorisation | `reports/TRAINING_CRNN_BIG.md` |
+
+**These two tables are not one protocol and must not be read as one ranking.** The foreign models
+were measured by the benchmark harness on 29 band-carrying types (580 items, 20 images per type);
+our two models were measured on their **own** held-out plugin splits with their own training
+pipelines — 7 types / 3 500 images and 4 types / 1 999 images respectively. Different type sets,
+different image counts, different pipelines: the numbers are not directly comparable.
+
 Every number in this card comes from a file of the **CaptchaOCR** project repository — the
 measured results are in `reports/`, they are not typed by hand and nothing is extrapolated.