| --- |
| license: mit |
| language: |
| - en |
| pipeline_tag: image-to-image |
| tags: |
| - denosing |
| - ImageEnhancement |
| --- |
| |
| # ImageDenosing |
|
|
| This is a collection of image denosing algorithms, models have been converted to run on the Axera NPU using **w8a8** quantization. |
|
|
| This model has been optimized with the following LoRA: |
|
|
| Compatible with Pulsar2 version: 6.0 115775d3 |
|
|
| ## Convert tools links: |
|
|
| For those who are interested in model conversion, you can try to export axmodel through |
|
|
| - [Pulsar2 Link, How to Convert ONNX to axmodel](https://pulsar2-docs.readthedocs.io/en/latest/pulsar2/introduction.html) |
|
|
|
|
| ## Support Platform |
|
|
| - AX650 |
| - [M4N-Dock(爱芯派Pro)](https://wiki.sipeed.com/hardware/zh/maixIV/m4ndock/m4ndock.html) |
| - [M.2 Accelerator card](https://docs.m5stack.com/en/ai_hardware/LLM-8850_Card) |
| |
|
|
| ## 性能基准测试 (Performance Benchmark) |
|
|
| | 模型 | 输入分辨率 | AX650板端耗时 | |
| |------|-----------|--------------| |
| | NAFNet | 256×256 | 87ms | |
| | DnCNN | 416×416 | 22ms | |
| | FFDNet | 640×640 | 19.3ms | |
| | FastDVDnet | 640×480 | 61.2ms | |
| | Restormer | 224×224 | 1600ms | |
|
|
| ## How to use |
|
|
| Download all files from this repository to the device |
|
|
| ``` |
| 模型文件组织方式如下： |
| . |
| |-- pic |
| | `-- noisy.png |
| |-- model_convert |
| | |-- NAFNet.json |
| | `-- axmodel |
| | `-- NAFNet_1_3_256_256.axmodel |
| |-- python |
| | |-- axmodel_infer.py |
| | `-- onnx_infer.py |
| `-- res |
| `-- axmodel_compare.png |
| |
| |
| ``` |
|
|
| ### Inference |
| #### Image Denoising |
|
|
| 图片推理，执行命令`python3 axmodel_infer.py`: |
| ``` |
| (base) root@ax650:~/NAFNet# python3 axmodel_infer.py |
| [INFO] Available providers: ['AxEngineExecutionProvider', 'AXCLRTExecutionProvider'] |
| [INFO] Using provider: AxEngineExecutionProvider |
| [INFO] Chip type: ChipType.MC50 |
| [INFO] VNPU type: VNPUType.DISABLED |
| [INFO] Engine version: 2.12.0s |
| [INFO] Model type: 2 (triple core) |
| [INFO] Compiler version: 6.0 115775d3 |
| axmodel inference finished: axmodel_compare.png |
| |
| ``` |
|
|
| 推理结果样例： |
|  |
|
|
| #### Video Denoising |
|
|
| 视频推理，执行命令`python3 axmodel_video_infer.py`: |
| ``` |
| (base) root@ax650:~/fastdvdnet# python3 axmodel_video_infer.py |
| [INFO] Available providers: ['AxEngineExecutionProvider', 'AXCLRTExecutionProvider'] |
| [INFO] Using provider: AxEngineExecutionProvider |
| [INFO] Chip type: ChipType.MC50 |
| [INFO] VNPU type: VNPUType.DISABLED |
| [INFO] Engine version: 2.12.0s |
| [INFO] Model type: 2 (triple core) |
| [INFO] Compiler version: 6.0 115775d3 |
| providers: AxEngineExecutionProvider |
| inputs: [('noisy', [1, 15, 480, 640], 'float32'), ('noise_map', [1, 1, 480, 640], 'float32')] |
| outputs: [('denoised', [1, 3, 480, 640])] |
| video: 854x480 @ 25.0fps, 67 frames |
| axmodel input: 640x480 |
| [OK] ./people-sunset.mp4 -> ./people-sunset_axmodel_side_by_side.mp4 |
| frames=67 time=17.57s fps=3.81 |
| |
| ``` |
|
|
| 视频降噪样例： |
|
|
|  |
