--- license: mit language: - en base_model: - PP-OCRv6_mobile_det - PP-OCRv6_mobile_rec - PP-LCNet_x0_25_textline_ori pipeline_tag: text-classification tags: - OCR - paddle - PPOCRv6 - axera --- # PPOCR_v6 > English | [中文](./README-zh.md) This version of PPOCR_v6 has been converted to run on AXERA NPU with **w8a16** quantization. ## Conversion Tool Links If you are interested in model conversion, you can export axmodel through the following links: - [ax-samples-github](https://github.com/AXERA-TECH/ax-samples), other interesting samples - [Pulsar2 Documentation, ONNX to axmodel conversion](https://pulsar2-docs.readthedocs.io/en/latest/pulsar2/introduction.html) ## Supported Platforms - AX650 - [M4N-Dock (AXERA Pi Pro)](https://wiki.sipeed.com/hardware/zh/maixIV/m4ndock/m4ndock.html) - [M.2 Accelerator Card](https://axcl-docs.readthedocs.io/zh-cn/latest/doc_guide_hardware.html) - AX630C - [AXERA Pi 2](https://axera-pi-2-docs-cn.readthedocs.io/zh-cn/latest/index.html) - [Module-LLM](https://docs.m5stack.com/zh_CN/module/Module-LLM) - [LLM630 Compute Kit](https://docs.m5stack.com/zh_CN/core/LLM630%20Compute%20Kit) - AX615 - [AX615 IPC SoC](https://www.axera-tech.com/zh-hans/product/2956.html) ### Performance Benchmarks | Chip | Model | npu_mode | Latency (ms) | | ----- | ------------------------------- | -------- | ------------ | | | PP-OCRv6_small_det | NPU1 | 42.158 | | | PP-OCRv6_small_det | NPU3 | 23.837 | | AX650 | PP-LCNet_x0_25_textline_ori | NPU1 | 0.294 | | | PP-LCNet_x0_25_textline_ori | NPU3 | 0.172 | | | PP-OCRv6_small_rec | NPU1 | 2.473 | | | PP-OCRv6_small_rec | NPU3 | 1.073 | | | - | - | - | | | PP-OCRv6_small_det | NPU1 | \ | | | PP-OCRv6_small_det | NPU2 | 186.738 | | AX630c| PP-LCNet_x0_25_textline_ori | NPU1 | 0.428 | | | PP-LCNet_x0_25_textline_ori | NPU2 | 0.381 | | | PP-OCRv6_small_rec | NPU1 | 25.251 | | | PP-OCRv6_small_rec | NPU2 | 9.535 | | | - | - | - | | | PP-OCRv6_small_det | NPU1 | \ | | | PP-OCRv6_small_det | NPU2 | 198.401 | | AX615 | PP-LCNet_x0_25_textline_ori | NPU1 | 0.851 | | | PP-LCNet_x0_25_textline_ori | NPU2 | 0.728 | | | PP-OCRv6_small_rec | NPU1 | 36.178 | | | PP-OCRv6_small_rec | NPU2 | 8.036 | Benchmarking command: ``` bash ax_run_model -w 10 -r 100 -m xx.axmodel ``` Recognition and detection ONNX model sources: [small-det-onnx](https://huggingface.co/PaddlePaddle/PP-OCRv6_small_det_onnx) and [small-rec-onnx](https://huggingface.co/PaddlePaddle/PP-OCRv6_small_rec_onnx) from [PaddlePaddle/PP-OCRv6](https://huggingface.co/collections/PaddlePaddle/pp-ocrv6) Text direction classifier: [AXERA-TECH/PPOCR_v5](https://huggingface.co/AXERA-TECH/PPOCR_v5/tree/main/onnx) ## Usage Download all files from this repository to your device. ``` PPOCR_v6# tree -L 1 . |-- 11.jpg # Test image |-- README-zh.md |-- README.md |-- axmodel # Axmodel files for each version |-- cls.json # Text direction classifier axmodel conversion config |-- dataset # Quantization dataset & test dataset |-- det.json # Detection model axmodel conversion config |-- fonts # Rendering fonts |-- onnx # Original ONNX files for each model |-- ppocrv6_ax.py # Axmodel inference pipeline |-- ppocrv6_onnx.py # ONNX inference pipeline |-- rec.json # Recognition model axmodel conversion config |-- res-ax.jpg # Axmodel inference result |-- res-onnx.jpg # ONNX inference result |-- run_det_ax.py # Detection axmodel accuracy test script |-- run_det_onnx.py # Detection ONNX accuracy test script |-- run_rec_ax.py # Recognition axmodel accuracy test script `-- run_rec_onnx.py # Recognition ONNX accuracy test script ``` ### Conversion ``` cd dataset sh download_quant_dataset.sh sh download_val_dataset.sh cd .. pulsar2 build --config det.json pulsar2 build --config cls.json pulsar2 build --config rec.json ``` ### Testing #### Detection ``` python python3 run_det_onnx.py --resize_mode letterbox # resize_mode options: letterbox (default), stretch (official) """ Images: 50 GT boxes: 201 DET boxes: 151 Matched: 77 Precision: 0.5099 (50.99%) Recall: 0.3831 (38.31%) Hmean (F1): 0.4375 """ python3 run_det_ax.py --resize_mode letterbox # resize_mode options: letterbox (default), stretch (official) """ Images: 50 GT boxes: 201 DET boxes: 150 Matched: 75 Precision: 0.5000 (50.00%) Recall: 0.3731 (37.31%) Hmean (F1): 0.4274 """ ``` Note: When `resize_mode` is `stretch`, it follows the official approach of directly resizing to the model input size. When `resize_mode` is `letterbox`, it pads the bottom-right corner, which has a smaller gap from the dynamic-input ONNX model and achieves better metrics in this test. You can download the `inference.onnx` with dynamic input `shape` from [small-det-onnx](https://huggingface.co/PaddlePaddle/PP-OCRv6_small_det_onnx) for comparison. #### Recognition ``` python python3 run_rec_onnx.py """ Total samples: 2077 Correct (exact match): 1563 Accuracy: 0.7525 (75.25%) Norm Edit Distance: 0.8947 """ python3 run_rec_ax.py """ Total samples: 2077 Correct (exact match): 1518 Accuracy: 0.7309 (73.09%) Norm Edit Distance: 0.8781 """ ``` ### Inference Run inference on AX650 host, such as M4N-Dock (AXERA Pi Pro). Input image: ![input](./11.jpg) ``` python python3 ppocrv6_onnx.py --use_angle_cls --visualize --image 11.jpg python3 ppocrv6_ax.py --use_angle_cls --visualize --image 11.jpg ``` Output image: ![output](./res-ax.jpg) ### Others The `tiny` recognition model has a large quantization error. Metrics are as follows: ``` onnx-preds: Total samples: 2077 Correct (exact match): 1271 Accuracy: 0.6119 (61.19%) Norm Edit Distance: 0.8263 ax-w8a16-preds: Total samples: 2077 Correct (exact match): 1178 Accuracy: 0.5672 (56.72%) Norm Edit Distance: 0.7941 ``` #### TODO - [x] ax630c performance benchmark - [x] ax615 performance benchmark