# Qwen2 with 1.5B ONNX INT8 — Mobile-Optimized Quantized Model

## 📌 Description
This is a quantized INT8 version of the Qwen language model, 
converted to ONNX format for efficient inference on low-resource 
devices such as mobile phones and edge hardware.

## 🎯 Objective
The goal of this project is to make Qwen accessible on devices 
with limited memory and compute power (e.g. iPhone 12, mid-range 
Android phones) without requiring high-end GPUs or cloud infrastructure.

## ⚡ Key Features
- **Format**: ONNX (INT8 quantized)
- **Size**: Reduced ~4x compared to FP32 original
- **Target hardware**: CPU-only devices, mobile, edge
- **Inference engine**: ONNX Runtime (`onnxruntime`)

## 🛠️ Usage

### Install dependencies
pip install onnxruntime numpy

### Run inference
import onnxruntime as ort

session = ort.InferenceSession(
    "model_int8.onnx",
    providers=["CPUExecutionProvider"]
)

## 📁 Files
| File | Description |
|---|---|
| `model.onnx` | Original FP32 model |
| `model_int8.onnx` | INT8 quantized model ✅ |
| `tokenizer.json` | Tokenizer config |
| `vocab.json` | Vocabulary |
| `quantize_int8.py` | Quantization script |

![image](https://cdn-uploads.huggingface.co/production/uploads/69b8999e75f123eb8e7e6776/BkHMV0hqyXoYfl1Oew2F2.png)