llmware
/

phi-3-mini-4k-instruct-onnx-qnn

4-bit precision

Model card Files Files and versions

phi-3-mini-4k-onnx-qnn

phi-3-mini-4k-onnx-qnn is an ONNX QNN int4 quantized version of Microsoft Phi-3-mini-instruct, providing a small fast NPU inference implementation, optimized for NPU deployment on Windows ARM64 AI PCs with Snapdragon Elite X NPU processors.

Model Description

Developed by: microsoft
Model type: phi3
Parameters: 3.8 billion
Model Parent: microsoft/Phi-3-mini-instruct
Language(s) (NLP): English
License: Apache 2.0
Uses: Chat, general-purpose LLM
Quantization: int4
Backend: qairt 2.36, ort 1.22.2, ortg 0.9

Model Card Contact

llmware website

Downloads last month: 23

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Collection including llmware/phi-3-mini-4k-instruct-onnx-qnn

NPU - QNN

leading models optimized for NPU deployment on Qualcomm Snapdragon • 7 items • Updated 2 days ago