| --- |
| license: mit |
| language: |
| - en |
| base_model: |
| - PP-OCRv6_mobile_det |
| - PP-OCRv6_mobile_rec |
| - PP-LCNet_x0_25_textline_ori |
| pipeline_tag: text-classification |
| tags: |
| - OCR |
| - paddle |
| - PPOCRv6 |
| - axera |
| --- |
| |
| # PPOCR_v6 |
| > English | [中文](./README-zh.md) |
| |
| This version of PPOCR_v6 has been converted to run on AXERA NPU with **w8a16** quantization. |
|
|
| ## Conversion Tool Links |
|
|
| If you are interested in model conversion, you can export axmodel through the following links: |
|
|
| - [ax-samples-github](https://github.com/AXERA-TECH/ax-samples), other interesting samples |
|
|
| - [Pulsar2 Documentation, ONNX to axmodel conversion](https://pulsar2-docs.readthedocs.io/en/latest/pulsar2/introduction.html) |
|
|
| ## Supported Platforms |
|
|
| - AX650 |
| - [M4N-Dock (AXERA Pi Pro)](https://wiki.sipeed.com/hardware/zh/maixIV/m4ndock/m4ndock.html) |
| - [M.2 Accelerator Card](https://axcl-docs.readthedocs.io/zh-cn/latest/doc_guide_hardware.html) |
| - AX630C |
| - [AXERA Pi 2](https://axera-pi-2-docs-cn.readthedocs.io/zh-cn/latest/index.html) |
| - [Module-LLM](https://docs.m5stack.com/zh_CN/module/Module-LLM) |
| - [LLM630 Compute Kit](https://docs.m5stack.com/zh_CN/core/LLM630%20Compute%20Kit) |
| - AX615 |
| - [AX615 IPC SoC](https://www.axera-tech.com/zh-hans/product/2956.html) |
|
|
| ### Performance Benchmarks |
|
|
| | Chip | Model | npu_mode | Latency (ms) | |
| | ----- | ------------------------------- | -------- | ------------ | |
| | | PP-OCRv6_small_det | NPU1 | 42.158 | |
| | | PP-OCRv6_small_det | NPU3 | 23.837 | |
| | AX650 | PP-LCNet_x0_25_textline_ori | NPU1 | 0.294 | |
| | | PP-LCNet_x0_25_textline_ori | NPU3 | 0.172 | |
| | | PP-OCRv6_small_rec | NPU1 | 2.473 | |
| | | PP-OCRv6_small_rec | NPU3 | 1.073 | |
| | | - | - | - | |
| | | PP-OCRv6_small_det | NPU1 | \ | |
| | | PP-OCRv6_small_det | NPU2 | 186.738 | |
| | AX630c| PP-LCNet_x0_25_textline_ori | NPU1 | 0.428 | |
| | | PP-LCNet_x0_25_textline_ori | NPU2 | 0.381 | |
| | | PP-OCRv6_small_rec | NPU1 | 25.251 | |
| | | PP-OCRv6_small_rec | NPU2 | 9.535 | |
| | | - | - | - | |
| | | PP-OCRv6_small_det | NPU1 | \ | |
| | | PP-OCRv6_small_det | NPU2 | 198.401 | |
| | AX615 | PP-LCNet_x0_25_textline_ori | NPU1 | 0.851 | |
| | | PP-LCNet_x0_25_textline_ori | NPU2 | 0.728 | |
| | | PP-OCRv6_small_rec | NPU1 | 36.178 | |
| | | PP-OCRv6_small_rec | NPU2 | 8.036 | |
| |
| Benchmarking command: |
| |
| ``` bash |
| ax_run_model -w 10 -r 100 -m xx.axmodel |
| ``` |
| |
| Recognition and detection ONNX model sources: [small-det-onnx](https://huggingface.co/PaddlePaddle/PP-OCRv6_small_det_onnx) and [small-rec-onnx](https://huggingface.co/PaddlePaddle/PP-OCRv6_small_rec_onnx) from [PaddlePaddle/PP-OCRv6](https://huggingface.co/collections/PaddlePaddle/pp-ocrv6) |
| |
| Text direction classifier: [AXERA-TECH/PPOCR_v5](https://huggingface.co/AXERA-TECH/PPOCR_v5/tree/main/onnx) |
| |
| ## Usage |
| |
| Download all files from this repository to your device. |
| |
| ``` |
| PPOCR_v6# tree -L 1 |
| . |
| |-- 11.jpg # Test image |
| |-- README-zh.md |
| |-- README.md |
| |-- axmodel # Axmodel files for each version |
| |-- cls.json # Text direction classifier axmodel conversion config |
| |-- dataset # Quantization dataset & test dataset |
| |-- det.json # Detection model axmodel conversion config |
| |-- fonts # Rendering fonts |
| |-- onnx # Original ONNX files for each model |
| |-- ppocrv6_ax.py # Axmodel inference pipeline |
| |-- ppocrv6_onnx.py # ONNX inference pipeline |
| |-- rec.json # Recognition model axmodel conversion config |
| |-- res-ax.jpg # Axmodel inference result |
| |-- res-onnx.jpg # ONNX inference result |
| |-- run_det_ax.py # Detection axmodel accuracy test script |
| |-- run_det_onnx.py # Detection ONNX accuracy test script |
| |-- run_rec_ax.py # Recognition axmodel accuracy test script |
| `-- run_rec_onnx.py # Recognition ONNX accuracy test script |
| ``` |
| |
| ### Conversion |
| ``` |
| cd dataset |
| sh download_quant_dataset.sh |
| sh download_val_dataset.sh |
| cd .. |
| pulsar2 build --config det.json |
| pulsar2 build --config cls.json |
| pulsar2 build --config rec.json |
|
|
| ``` |
| |
| ### Testing |
| |
| #### Detection |
| |
| ``` python |
| python3 run_det_onnx.py --resize_mode letterbox # resize_mode options: letterbox (default), stretch (official) |
| """ |
| Images: 50 |
| GT boxes: 201 |
| DET boxes: 151 |
| Matched: 77 |
| Precision: 0.5099 (50.99%) |
| Recall: 0.3831 (38.31%) |
| Hmean (F1): 0.4375 |
| """ |
| python3 run_det_ax.py --resize_mode letterbox # resize_mode options: letterbox (default), stretch (official) |
| """ |
| Images: 50 |
| GT boxes: 201 |
| DET boxes: 150 |
| Matched: 75 |
| Precision: 0.5000 (50.00%) |
| Recall: 0.3731 (37.31%) |
| Hmean (F1): 0.4274 |
| """ |
| ``` |
| |
| Note: When `resize_mode` is `stretch`, it follows the official approach of directly resizing to the model input size. When `resize_mode` is `letterbox`, it pads the bottom-right corner, which has a smaller gap from the dynamic-input ONNX model and achieves better metrics in this test. You can download the `inference.onnx` with dynamic input `shape` from [small-det-onnx](https://huggingface.co/PaddlePaddle/PP-OCRv6_small_det_onnx) for comparison. |
| |
| #### Recognition |
| |
| ``` python |
| python3 run_rec_onnx.py |
| """ |
| Total samples: 2077 |
| Correct (exact match): 1563 |
| Accuracy: 0.7525 (75.25%) |
| Norm Edit Distance: 0.8947 |
| """ |
| python3 run_rec_ax.py |
| """ |
| Total samples: 2077 |
| Correct (exact match): 1518 |
| Accuracy: 0.7309 (73.09%) |
| Norm Edit Distance: 0.8781 |
| """ |
| ``` |
| |
| ### Inference |
| |
| Run inference on AX650 host, such as M4N-Dock (AXERA Pi Pro). |
| |
| Input image: |
|  |
| |
| |
| ``` python |
| python3 ppocrv6_onnx.py --use_angle_cls --visualize --image 11.jpg |
| python3 ppocrv6_ax.py --use_angle_cls --visualize --image 11.jpg |
| ``` |
| |
| Output image: |
|  |
| |
| ### Others |
| The `tiny` recognition model has a large quantization error. Metrics are as follows: |
| ``` |
| onnx-preds: |
| Total samples: 2077 |
| Correct (exact match): 1271 |
| Accuracy: 0.6119 (61.19%) |
| Norm Edit Distance: 0.8263 |
|
|
| ax-w8a16-preds: |
| Total samples: 2077 |
| Correct (exact match): 1178 |
| Accuracy: 0.5672 (56.72%) |
| Norm Edit Distance: 0.7941 |
| ``` |
| |
| #### TODO |
| |
| - [x] ax630c performance benchmark |
| - [x] ax615 performance benchmark |
| |
