Update README.md

README.md

@@ -1,3 +1,131 @@
----
-license: mit
----
+---
+license: mit
+language:
+- en
+tags:
+- Transformer
+- Tracking
+- ONNX
+---
+
+# MixFormerV2
+
+This version of MixFormerV2 has been converted to run on the Axera NPU using **w8a16** quantization.
+
+This model has been optimized with the following LoRA: 
+
+Compatible with Pulsar2 version: 3.4
+
+## Convert tools links:
+
+For those who are interested in model conversion, you can try to export axmodel through 
+
+- [The repo of original](https://github.com/MCG-NJU/MixFormerV2)
+
+- [The repo of AXera Platform](https://github.com/Jordan-5i/ax650_mixformer2_demo), which you can get the detial of guide
+
+- [Pulsar2 Link, How to Convert ONNX to axmodel](https://pulsar2-docs.readthedocs.io/en/latest/pulsar2/introduction.html) 
+
+
+## Support Platform
+
+- AX650
+  - [M4N-Dock(爱芯派Pro)](https://wiki.sipeed.com/hardware/zh/maixIV/m4ndock/m4ndock.html)
+  - [M.2 Accelerator card](https://axcl-docs.readthedocs.io/zh-cn/latest/doc_guide_hardware.html)
+- AX630C(todo, convert model failed, try to fix......)
+  
+ 
+|Chips|npu1(6TOPs)|npu3(18TOPs)|
+|--|--|--|
+|AX650| 11 ms | TBD |
+
+## How to use
+
+Download all files from this repository to the device
+
+```
+root@ax650:/mnt/qtang/ax650_mixformer2_demo# tree -L 1
+.
+├── README.md
+├── ax650
+├── car.avi
+├── config.json
+├── onnx
+├── run_mixformer2_axmodel.py
+└── run_mixformer2_onnx.py
+```
+
+### python env requirement
+
+#### pyaxengine
+
+https://github.com/AXERA-TECH/pyaxengine
+
+```
+wget https://github.com/AXERA-TECH/pyaxengine/releases/download/0.1.1rc0/axengine-0.1.1-py3-none-any.whl
+pip install axengine-0.1.1-py3-none-any.whl
+```
+
+#### others
+
+```
+pip install argparse numpy opencv-python glob2
+```
+
+#### Inference with AX650 Host, such as M4N-Dock(爱芯派Pro)
+
+```
+root@ax650:/mnt/qtang/ax650_mixformer2_demo# python3 run_mixformer2_axmodel.py --model-path ax650/mixformer_v2.axmodel --frame-path car.avi -r 10
+[INFO] Available providers:  ['AxEngineExecutionProvider']
+[INFO] Using provider: AxEngineExecutionProvider
+[INFO] Chip type: ChipType.MC50
+[INFO] VNPU type: VNPUType.DISABLED
+[INFO] Engine version: 2.7.2a
+[INFO] Model type: 0 (single core)
+[INFO] Compiler version: 3.4-dirty 4ff37520-dirty
+====================type================= [1079, 482] <class 'list'> <class 'list'>
+第一帧初始化完毕！
+Video: tracking     246.0fps
+Video: tracking     4.0fps
+Video: tracking     4.0fps
+Video: tracking     4.0fps
+Video: tracking     4.0fps
+Video: tracking     4.0fps
+Video: tracking     4.0fps
+Video: tracking     4.0fps
+Video: tracking     4.0fps
+Video: tracking     4.0fps
+Video: tracking     4.0fps
+Reached the maximum number of frames (10). Exiting loop.
+video: average finale average tracking fps 31.8 fps
+root@ax650:/mnt/qtang/ax650_mixformer2_demo#
+```
+
+#### Inference with M.2 Accelerator card
+
+[What is M.2 Accelerator card?](https://axcl-docs.readthedocs.io/zh-cn/latest/doc_guide_hardware.html), Show this DEMO based on Raspberry PI 5.
+
+```
+(axcl) axera@raspberrypi:~/samples/MixFormerV2 $ python3 run_mixformer2_axmodel.py --model-path ax650/mixformer_v2.axmodel --frame-path car.avi -r 10
+[INFO] Available providers:  ['AXCLRTExecutionProvider']
+[INFO] Using provider: AXCLRTExecutionProvider
+[INFO] SOC Name: AX650N
+[INFO] VNPU type: VNPUType.DISABLED
+[INFO] Compiler version: 3.4-dirty 4ff37520-dirty
+====================type================= [1079, 482] <class 'list'> <class 'list'>
+第一帧初始化完毕！
+Video: tracking     925.0fps
+Video: tracking     12.0fps
+Video: tracking     12.0fps
+Video: tracking     11.0fps
+Video: tracking     11.0fps
+Video: tracking     11.0fps
+Video: tracking     11.0fps
+Video: tracking     11.0fps
+Video: tracking     10.0fps
+Video: tracking     10.0fps
+Video: tracking     10.0fps
+Reached the maximum number of frames (10). Exiting loop.
+video: average finale average tracking fps 114.9 fps
+(axcl) axera@raspberrypi:~/samples/MixFormerV2 $
+```