---
license: mit
language:
- en
base_model:
- KwaiVGI/LivePortrait
pipeline_tag: image-to-video
---
# LivePortrait
This version of LivePortrait has been converted to run on the Axera NPU using **w8a16** quantization.
This model has been optimized with the following:
Compatible with Pulsar2 version: 3.4
## Convert tools links:
For those who are interested in model conversion:
- [the original repo](https://huggingface.co/KwaiVGI/LivePortrait)
- [Github for LivePortrait](https://github.com/AXERA-TECH/LivePortrait.axera)
## Support Platform
- AX650/AX8850
- [M4N-Dock(爱芯派Pro)](https://wiki.sipeed.com/hardware/zh/maixIV/m4ndock/m4ndock.html)
- [M.2 Accelerator card](https://axcl-docs.readthedocs.io/zh-cn/latest/doc_guide_hardware.html)
## How to use
Download all files from this repository to the device.
```
(py310) axera@dell:~/samples/LivePortrait$ tree -L 2
.
├── assets
│ └── examples
├── config.json
├── python
│ ├── axmodels
│ ├── cropper.py
│ ├── infer_onnx.py
│ ├── infer.py
│ ├── pretrained_weights
│ ├── requirements.txt
│ └── utils
└── README.md
7 directories, 6 files
```
### python env requirement
#### pyaxengine
https://github.com/AXERA-TECH/pyaxengine
```
wget https://github.com/AXERA-TECH/pyaxengine/releases/download/0.1.3.rc1/axengine-0.1.3-py3-none-any.whl
pip install axengine-0.1.3-py3-none-any.whl
```
#### others
```
pip install -r python/requirements.txt
```
## Inference with AX650 or AX8850 Host, such as AX650 DEMO BOARD, M4N-DOCK(爱芯派Pro)
```
root@ax650 ~/yongqiang/LivePortrait.axera # python3 ./python/infer.py --source ./assets/examples/source/s0.jpg --driving ./assets/examples/driving/d8.jpg --models ./python/axmodels/ --output-dir ./axmodel_infer
[INFO] Available providers: ['AxEngineExecutionProvider']
[INFO] Using provider: AxEngineExecutionProvider
[INFO] Chip type: ChipType.MC50
[INFO] VNPU type: VNPUType.DISABLED
[INFO] Engine version: 2.12.0s
[INFO] Model type: 2 (triple core)
[INFO] Compiler version: 3.3 144960ad
[INFO] Using provider: AxEngineExecutionProvider
[INFO] Model type: 2 (triple core)
[INFO] Compiler version: 3.3 144960ad
[INFO] Using provider: AxEngineExecutionProvider
[INFO] Model type: 2 (triple core)
[INFO] Compiler version: 3.3 0f7260e8
[INFO] Using provider: AxEngineExecutionProvider
[INFO] Model type: 2 (triple core)
[INFO] Compiler version: 3.3 144960ad
FaceAnalysisDIY warmup time: 0.598s
LandmarkRunner warmup time: 0.769s
2025-05-30 09:56:12.247 | INFO | __main__:main:727 - Start making driving motion template...
2025-05-30 09:56:14.770 | INFO | __main__:main:747 - Prepared pasteback mask done.
2025-05-30 09:56:17.219 | INFO | __main__:main:787 - The output of image-driven portrait animation is an image.
2025-05-30 09:56:30.701 | DEBUG | __main__:warp_decode:647 - warp time: 13.475s
2025-05-30 09:56:31.118 | INFO | __main__:main:881 - Animated image: ./axmodel_infer/s0--d8.jpg
2025-05-30 09:56:31.118 | INFO | __main__:main:882 - Animated image with concat: ./axmodel_infer/s0--d8_concat.jpg
2025-05-30 09:56:31.167 | DEBUG | __main__::894 - LivePortrait axmodel infer time: 32.455s
```
## Inference with M.2 Accelerator card
[What is M.2 Accelerator card?](https://axcl-docs.readthedocs.io/zh-cn/latest/doc_guide_hardware.html), Show this DEMO based on x86.
### Image
```
(py310) axera@dell:~/samples/LivePortrait$ python ./python/infer.py --source ./assets/examples/source/s0.jpg --driving ./assets/examples/driving/d8.jpg --models ./python/axmodels/ --output-dir ./axmodel_infer
[INFO] Available providers: ['AXCLRTExecutionProvider']
[INFO] Using provider: AXCLRTExecutionProvider
[INFO] SOC Name: AX650N
[INFO] VNPU type: VNPUType.DISABLED
[INFO] Compiler version: 3.3 144960ad
[INFO] Using provider: AXCLRTExecutionProvider
[INFO] SOC Name: AX650N
[INFO] VNPU type: VNPUType.DISABLED
[INFO] Compiler version: 3.3 144960ad
[INFO] Using provider: AXCLRTExecutionProvider
[INFO] SOC Name: AX650N
[INFO] VNPU type: VNPUType.DISABLED
[INFO] Compiler version: 3.3 0f7260e8
[INFO] Using provider: AXCLRTExecutionProvider
[INFO] SOC Name: AX650N
[INFO] VNPU type: VNPUType.DISABLED
[INFO] Compiler version: 3.3 144960ad
FaceAnalysisDIY warmup time: 0.024s
[20:02:20] LandmarkRunner warmup time: 0.031s human_landmark_runner.py:95
2025-05-29 20:02:20.727 | INFO | __main__:main:727 - Start making driving motion template...
2025-05-29 20:02:20.972 | INFO | __main__:main:747 - Prepared pasteback mask done.
2025-05-29 20:02:21.449 | INFO | __main__:main:787 - The output of image-driven portrait animation is an image.
2025-05-29 20:02:25.475 | DEBUG | __main__:warp_decode:647 - warp time: 4.017s
2025-05-29 20:02:25.892 | INFO | __main__:main:881 - Animated image: ./axmodel_infer/s0--d8.jpg
2025-05-29 20:02:25.892 | INFO | __main__:main:882 - Animated image with concat: ./axmodel_infer/s0--d8_concat.jpg
2025-05-29 20:02:25.904 | DEBUG | __main__::894 - LivePortrait axmodel infer time: 8.165s
(py310) axera@dell:~/samples/LivePortrait$
```
Here, `--models` specifies the storage path for the `*.axmodel model`.
The output of axmodel-infer is as follows:
### Video
```
python3 ./python/infer.py --source ./assets/examples/source/s0.jpg --driving ./assets/examples/driving/d0.mp4 --models ./python/axmodels/ --output-dir ./axmodel_infer
```
The output of `axmodel-infer` is as follows:
