Upload README.md with huggingface_hub

README.md

@@ -5,17 +5,19 @@ tags:
 - text-detection
 - scene-text-detection
 - mmocr
-- monkeyocr
+- monkeyocr v2
 ---
 
 # MonkeyOCRv2 Detection
 
-MonkeyOCRv2 Detection integrates the visual encoder from
+This repository provides the text detection experiments from the
+[MonkeyOCRv2 paper](https://arxiv.org/abs/2607.11562). The visual encoder from
 [MonkeyOCRv2-AS](https://huggingface.co/zenosai/MonkeyOCRv2-AS) (ViTAEv2-S,
-21M parameters) into **DBNet** and **PSENet** scene text detectors via
-[MMOCR](https://github.com/open-mmlab/mmocr). The four-stage ViTAEv2 features
-(strides 4/8/16/32) are exposed as `res2`–`res5` and fed to the standard FPNC /
-FPNF necks, so no change to the detection heads is required.
+21M parameters) is integrated into **DBNet** and **PSENet** scene text
+detectors via [MMOCR](https://github.com/open-mmlab/mmocr). The four-stage
+ViTAEv2 features (strides 4/8/16/32) are exposed as `res2`–`res5` and fed to
+the standard FPNC / FPNF necks, so no change to the detection heads is
+required.
 
 Training and evaluation follow the official MMOCR protocols on Total-Text,
 CTW1500, and ICDAR2015.
@@ -24,42 +26,41 @@ CTW1500, and ICDAR2015.
 
 Replacing the ImageNet-pretrained ResNet-50 with the MonkeyOCRv2-AS encoder
 consistently improves F-score over both the baseline and the
-[oCLIP](https://github.com/alkan25/oclip)-pretrained backbone. `*` marks
-results reproduced by us with MMOCR.
+[oCLIP](https://github.com/bytedance/oclip)-pretrained backbone.
 
 ### Total-Text
 
-| Method | P | R | F |
-| --- | ---: | ---: | ---: |
-| DBNet* (ResNet-50) | 82.6 | 78.4 | 80.4 |
-| DBNet + oCLIP | 85.1 | 81.7 | 83.4 |
+| Method                  |        P |    R |        F |
+| ----------------------- | -------: | ---: | -------: |
+| DBNet (ResNet-50)       |     82.6 | 78.4 |     80.4 |
+| DBNet + oCLIP           |     85.1 | 81.7 |     83.4 |
 | **DBNet + MonkeyOCRv2** | **87.7** | 80.1 | **83.7** |
 
 ### CTW1500
 
-| Method | P | R | F |
-| --- | ---: | ---: | ---: |
-| PSENet* (ResNet-50) | 80.1 | 82.7 | 81.4 |
-| PSENet + oCLIP | 82.1 | 85.5 | 83.8 |
+| Method                   |        P |    R |        F |
+| ------------------------ | -------: | ---: | -------: |
+| PSENet (ResNet-50)       |     80.1 | 82.7 |     81.4 |
+| PSENet + oCLIP           |     82.1 | 85.5 |     83.8 |
 | **PSENet + MonkeyOCRv2** | **88.3** | 82.0 | **85.1** |
 
 ### ICDAR2015
 
-| Method | P | R | F |
-| --- | ---: | ---: | ---: |
-| PSENet* (ResNet-50) | 84.0 | 76.2 | 79.9 |
-| PSENet + oCLIP | 87.3 | 82.6 | 84.9 |
-| **PSENet + MonkeyOCRv2** | **90.4** | 80.3 | **85.0** |
-| DBNet* (ResNet-50) | 88.8 | 81.5 | 85.0 |
-| DBNet + oCLIP | 90.9 | 84.1 | 87.4 |
-| **DBNet + MonkeyOCRv2** | **91.2** | **86.0** | **88.5** |
+| Method                   |        P |        R |        F |
+| ------------------------ | -------: | -------: | -------: |
+| PSENet (ResNet-50)       |     84.0 |     76.2 |     79.9 |
+| PSENet + oCLIP           |     87.3 |     82.6 |     84.9 |
+| **PSENet + MonkeyOCRv2** | **90.4** |     80.3 | **85.0** |
+| DBNet (ResNet-50)        |     88.8 |     81.5 |     85.0 |
+| DBNet + oCLIP            |     90.9 |     84.1 |     87.4 |
+| **DBNet + MonkeyOCRv2**  | **91.2** | **86.0** | **88.5** |
 
 ### Checkpoints
 
 Download the checkpoints from
-[HB16888/MonkeyOCRv2_det](https://huggingface.co/HB16888/MonkeyOCRv2_det)
+[HB16888/MonkeyOCRv2\_det](https://huggingface.co/HB16888/MonkeyOCRv2_det)
 (HuggingFace) or
-[WangXinhan/MonkeyOCRv2_det](https://modelscope.cn/models/WangXinhan/MonkeyOCRv2_det)
+[WangXinhan/MonkeyOCRv2\_det](https://modelscope.cn/models/WangXinhan/MonkeyOCRv2_det)
 (ModelScope):
 
 ```bash
@@ -72,20 +73,20 @@ modelscope download --model WangXinhan/MonkeyOCRv2_det --local_dir ./model_weigh
 Each checkpoint is the best epoch on the test set, i.e. exactly the row
 reported in the tables above.
 
-| Checkpoint | Method | Dataset | Epoch | Config |
-| --- | --- | --- | ---: | --- |
-| dbnet_r50_totaltext.pth | DBNet baseline | Total-Text | 580 | configs/textdet/dbnet/dbnet_resnet50_1200e_totaltext_2gpu.py |
-| dbnet_r50-oclip_totaltext.pth | DBNet + oCLIP | Total-Text | 740 | configs/textdet/dbnet/dbnet_resnet50-oclip_1200e_totaltext_2gpu.py |
-| dbnet_mkv2vitae_totaltext.pth | DBNet + MonkeyOCRv2 | Total-Text | 1000 | configs/textdet/dbnet/dbnet_mkv2vitae_1200e_totaltext_2gpu_adamw.py |
-| psenet_r50_ctw1500.pth | PSENet baseline | CTW1500 | 280 | configs/textdet/psenet/psenet_resnet50_fpnf_600e_ctw1500_2gpu.py |
-| psenet_r50-oclip_ctw1500.pth | PSENet + oCLIP | CTW1500 | 280 | configs/textdet/psenet/psenet_resnet50-oclip_fpnf_600e_ctw1500_2gpu.py |
-| psenet_mkv2vitae_ctw1500.pth | PSENet + MonkeyOCRv2 | CTW1500 | 120 | configs/textdet/psenet/psenet_mkv2vitae_fpnf_600e_ctw1500_4gpu_adamw.py |
-| psenet_r50_icdar2015.pth | PSENet baseline | ICDAR2015 | 400 | configs/textdet/psenet/psenet_resnet50_fpnf_600e_icdar2015_2gpu.py |
-| psenet_r50-oclip_icdar2015.pth | PSENet + oCLIP | ICDAR2015 | 520 | configs/textdet/psenet/psenet_resnet50-oclip_fpnf_600e_icdar2015_2gpu.py |
-| psenet_mkv2vitae_icdar2015.pth | PSENet + MonkeyOCRv2 | ICDAR2015 | 160 | configs/textdet/psenet/psenet_mkv2vitae_fpnf_600e_icdar2015_4gpu_adamw.py |
-| dbnet_r50_icdar2015.pth | DBNet baseline | ICDAR2015 | 980 | configs/textdet/dbnet/dbnet_resnet50_1200e_icdar2015_2gpu.py |
-| dbnet_r50-oclip_icdar2015.pth | DBNet + oCLIP | ICDAR2015 | 1100 | configs/textdet/dbnet/dbnet_resnet50-oclip_1200e_icdar2015_2gpu.py |
-| dbnet_mkv2vitae_icdar2015.pth | DBNet + MonkeyOCRv2 | ICDAR2015 | 420 | configs/textdet/dbnet/dbnet_mkv2vitae_1200e_icdar2015_2gpu_adamw.py |
+| Checkpoint                       | Method               | Dataset    | Epoch | Config                                                                           |
+| -------------------------------- | -------------------- | ---------- | ----: | -------------------------------------------------------------------------------- |
+| dbnet\_r50\_totaltext.pth        | DBNet baseline       | Total-Text |   580 | configs/textdet/dbnet/dbnet\_resnet50\_1200e\_totaltext\_2gpu.py                 |
+| dbnet\_r50-oclip\_totaltext.pth  | DBNet + oCLIP        | Total-Text |   740 | configs/textdet/dbnet/dbnet\_resnet50-oclip\_1200e\_totaltext\_2gpu.py           |
+| dbnet\_mkv2vitae\_totaltext.pth  | DBNet + MonkeyOCRv2  | Total-Text |  1000 | configs/textdet/dbnet/dbnet\_mkv2vitae\_1200e\_totaltext\_2gpu\_adamw\.py        |
+| psenet\_r50\_ctw1500.pth         | PSENet baseline      | CTW1500    |   280 | configs/textdet/psenet/psenet\_resnet50\_fpnf\_600e\_ctw1500\_2gpu.py            |
+| psenet\_r50-oclip\_ctw1500.pth   | PSENet + oCLIP       | CTW1500    |   280 | configs/textdet/psenet/psenet\_resnet50-oclip\_fpnf\_600e\_ctw1500\_2gpu.py      |
+| psenet\_mkv2vitae\_ctw1500.pth   | PSENet + MonkeyOCRv2 | CTW1500    |   120 | configs/textdet/psenet/psenet\_mkv2vitae\_fpnf\_600e\_ctw1500\_4gpu\_adamw\.py   |
+| psenet\_r50\_icdar2015.pth       | PSENet baseline      | ICDAR2015  |   400 | configs/textdet/psenet/psenet\_resnet50\_fpnf\_600e\_icdar2015\_2gpu.py          |
+| psenet\_r50-oclip\_icdar2015.pth | PSENet + oCLIP       | ICDAR2015  |   520 | configs/textdet/psenet/psenet\_resnet50-oclip\_fpnf\_600e\_icdar2015\_2gpu.py    |
+| psenet\_mkv2vitae\_icdar2015.pth | PSENet + MonkeyOCRv2 | ICDAR2015  |   160 | configs/textdet/psenet/psenet\_mkv2vitae\_fpnf\_600e\_icdar2015\_4gpu\_adamw\.py |
+| dbnet\_r50\_icdar2015.pth        | DBNet baseline       | ICDAR2015  |   980 | configs/textdet/dbnet/dbnet\_resnet50\_1200e\_icdar2015\_2gpu.py                 |
+| dbnet\_r50-oclip\_icdar2015.pth  | DBNet + oCLIP        | ICDAR2015  |  1100 | configs/textdet/dbnet/dbnet\_resnet50-oclip\_1200e\_icdar2015\_2gpu.py           |
+| dbnet\_mkv2vitae\_icdar2015.pth  | DBNet + MonkeyOCRv2  | ICDAR2015  |   420 | configs/textdet/dbnet/dbnet\_mkv2vitae\_1200e\_icdar2015\_2gpu\_adamw\.py        |
 
 ## Environment
 
@@ -102,25 +103,6 @@ This directory is an add-on on top of the official MMOCR v1.0.1. Run:
 bash install.sh            # clones MMOCR v1.0.1 into ./mmocr and patches it
 ```
 
-The add-on contains:
-
-- `mmocr/models/common/backbones/monkeyocr_v2_vitae.py` — MMEngine wrapper
-  that loads the HuggingFace MonkeyOCRv2-AS encoder (via `transformers`
-  `AutoModel`, `trust_remote_code=True`) and exposes its four stages as
-  `res2`–`res5`. Image normalization (ViTAE mean/std) and patch splitting are
-  performed inside the backbone; the data preprocessor must use
-  `mean=None`, `std=None`, `bgr_to_rgb=True`, `pad_size_divisor=32`.
-- `mmocr/models/textdet/postprocessors/pse_postprocessor.py` — one-line fix
-  for the PSE kernel-growing condition (`or` → `and` when filtering by
-  `score_threshold`), which improves the PSENet postprocessing quality.
-- `dataset_zoo/ctw1500/textdet.py` — updated MD5 sums and archive layouts
-  for the CTW1500 download sources (the official Box/CloudStor archives were
-  re-uploaded and no longer match the upstream checksums).
-- `tools/test.py` — accepts `--out` when `test_evaluator` is a plain dict.
-- `tools/dataset_converters/prepare_all_datasets.sh` — prepares ICDAR2015,
-  CTW1500, and Total-Text in one go.
-- `configs/textdet/{dbnet,psenet}/` — the 12 training configs listed above.
-
 ## Pretrained Backbone
 
 Download the MonkeyOCRv2-AS visual encoder before training or evaluation:
@@ -129,10 +111,6 @@ Download the MonkeyOCRv2-AS visual encoder before training or evaluation:
 hf download zenosai/MonkeyOCRv2-AS --local-dir ./pretrained/monkeyocrv2_as
 ```
 
-The configs reference `pretrained/monkeyocrv2_as` relative to the MMOCR
-working directory. The weights are identical to the ones used to produce the
-results above (verified by MD5).
-
 ## Datasets
 
 ```bash
@@ -196,15 +174,6 @@ bash tools/dist_train.sh configs/textdet/dbnet/dbnet_resnet50-oclip_1200e_icdar2
 bash tools/dist_train.sh configs/textdet/dbnet/dbnet_mkv2vitae_1200e_icdar2015_2gpu_adamw.py 2
 ```
 
-Notes:
-
-- The oCLIP backbone weights (`resnet50-oclip-7ba0c533.pth`) are fetched
-  automatically from `download.openmmlab.com` via `init_cfg`.
-- The MonkeyOCRv2 configs load the encoder from
-  `pretrained/monkeyocrv2_as` (see [Pretrained Backbone](#pretrained-backbone)).
-- Every config sets `randomness = dict(seed=42)`; results are selected by the
-  best test-set hmean over the training run, evaluated every 20 epochs.
-
 ## Evaluation
 
 ```bash
@@ -225,6 +194,7 @@ The evaluation prints `precision / recall / hmean` with the
 ## Acknowledgements
 
 This project builds on [MMOCR](https://github.com/open-mmlab/mmocr),
-[DBNet](https://arxiv.org/abs/1911.08947),
-[PSENet](https://arxiv.org/abs/1806.02559), and
-[oCLIP](https://arxiv.org/abs/2303.06995).
+[DBNet](https://github.com/MhLiao/DB),
+[PSENet](https://github.com/whai362/PSENet),
+[oCLIP](https://github.com/bytedance/oclip), and
+[MonkeyOCRv2](https://github.com/Yuliang-Liu/MonkeyOCRv2).