| run phi3-mini on AMD NPU | |
| 1. If no ```phi3_mini_awq_4bit_no_flash_attention.pt```, use awq quantization to get the quantization model. | |
| 2. Put modeling_phi3.py in this repo into the phi-3-mini folder. | |
| 3. Modify the file path in the run_awq.py | |
| 4. run ```python run_awq.py --task decode --target aie --w_bit 4``` | |
| reference:https://github.com/amd/RyzenAI-SW/tree/main/example/transformers | |
| As the quantization of phi-3, may refer to https://github.com/mit-han-lab/llm-awq/pull/183 | |
| PS: The performance is similar to that on CPU(7640hs). |