README.md

---
language:
  - en
license: apache-2.0
base_model: Irfanuruchi/Qwen3-4B-Computer-Science
base_model_relation: quantized
library_name: optimum-intel
pipeline_tag: text-generation
inference: false
tags:
  - qwen3
  - computer-science
  - software-engineering
  - programming
  - coding
  - openvino
  - optimum-intel
  - int4
  - quantized
  - transformers
  - cpu
  - gpu
  - npu
---

# Qwen3-4B-Computer-Science OpenVINO INT4

This repository provides an **OpenVINO INT4** deployment of **Qwen3-4B-Computer-Science**, optimized for efficient inference using **Intel® OpenVINO™** and **Optimum Intel**.

The model is designed for software engineering and computer science workloads while significantly reducing storage requirements and memory usage through INT4 weight compression. It enables efficient deployment across OpenVINO-supported hardware while preserving the capabilities of the original model.

---

# Model Information

| Property | Value |
|----------|-------|
| Base Model | Irfanuruchi/Qwen3-4B-Computer-Science |
| Base Architecture | Qwen3 |
| Parameters | ~4 Billion |
| Deployment Format | OpenVINO IR |
| Weight Compression | INT4 Asymmetric |
| Compression Group Size | 128 |
| Runtime | Intel® OpenVINO™ Runtime |
| Library | Optimum Intel |
| Supported Hardware | OpenVINO-supported devices* |
| License | Apache License 2.0 |

> **\*** Supported execution devices depend on the installed OpenVINO Runtime, operating system, drivers, and available hardware. Depending on the platform, inference may be executed on supported CPUs, integrated GPUs, NPUs, or other OpenVINO-compatible accelerators.

---

# Features

- OpenVINO IR deployment format
- INT4 asymmetric weight compression
- Reduced storage footprint
- Lower memory usage
- Efficient inference on OpenVINO-supported hardware
- Compatible with Optimum Intel and Hugging Face Transformers
- Exported tokenizer and detokenizer
- Chat template included
- Ready for local deployment

---

# Export Configuration

The model was exported using:

```bash
optimum-cli export openvino \
    --model Irfanuruchi/Qwen3-4B-Computer-Science \
    --task text-generation-with-past \
    --weight-format int4 \
    Qwen3-4B-Computer-Science-OpenVINO-INT4
```

Compression summary:

- **252 transformer layers** compressed using **INT4 asymmetric** with **group size 128**
- **1 auxiliary layer** stored using **INT8 per-channel**

---

# Installation

```bash
pip install -U openvino optimum-intel transformers
```

---

# Example Usage

```python
from transformers import AutoTokenizer
from optimum.intel.openvino import OVModelForCausalLM

model_id = "Irfanuruchi/Qwen3-4B-Computer-Science-OpenVINO-INT4"

tokenizer = AutoTokenizer.from_pretrained(model_id)

model = OVModelForCausalLM.from_pretrained(
    model_id,
    device="CPU",
)

messages = [
    {
        "role": "system",
        "content": "You are a computer science assistant."
    },
    {
        "role": "user",
        "content": "Explain Floyd's cycle detection algorithm."
    },
]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)

inputs = tokenizer(prompt, return_tensors="pt")

outputs = model.generate(
    **inputs,
    max_new_tokens=256,
    do_sample=False,
)

response = tokenizer.decode(
    outputs[0][inputs["input_ids"].shape[1]:],
    skip_special_tokens=True,
)

print(response)
```

---

# Validation

The exported model was successfully validated using:

- Intel® OpenVINO™ Runtime 2026.2.1
- Local OpenVINO inference
- Chat template support
- Greedy decoding
- Software engineering benchmark prompts

Validation confirmed successful generation of technically correct programming responses, including algorithm implementation and complexity analysis.

---

# Intended Use

This model is intended for:

- Software engineering assistance
- Computer science education
- Programming support
- Code generation
- Code review
- Debugging
- Algorithm design
- Technical documentation
- Technical question answering

---

# Model Family

The **Qwen3-4B-Computer-Science** release family currently includes:

| Model | Status |
|------|:------:|
| Transformers BF16 | ✅ |
| GGUF (Multiple Quantization Variants) | ✅ |
| AWQ | ✅ |
| MLX 4-bit | ✅ |
| MLX 8-bit | ✅ |
| MLX BF16 | ✅ |
| OpenVINO INT4 | ✅ |

Each release is maintained in its own repository with runtime-specific documentation, usage examples, integrity verification files, and configuration tailored to its target inference backend.

---

# Credits

This release builds upon the work of several open-source projects and communities:

- **Qwen Team** for the Qwen3 foundation model architecture.
- **Intel® OpenVINO™ Toolkit** for the OpenVINO Runtime and deployment framework.
- **Hugging Face** for the Transformers ecosystem and the Optimum Intel integration.
- **Optimum Intel** for OpenVINO model export and runtime integration.

This repository provides an **OpenVINO INT4 deployment** of the original **Qwen3-4B-Computer-Science** model for efficient inference on OpenVINO-supported hardware.

---

# Limitations

Like other large language models, this model may occasionally:

- Generate incorrect or incomplete code
- Hallucinate APIs or implementation details
- Produce inefficient implementations
- Misinterpret ambiguous instructions

INT4 weight compression may introduce minor quality differences compared to higher-precision variants.

All generated code should be reviewed and tested before use in production or safety-critical environments.

---

# License

This repository is distributed under the **Apache License 2.0**.

This release is an OpenVINO INT4 conversion of the original **Qwen3-4B-Computer-Science** model and retains the licensing and attribution requirements applicable to the original project.

See the included **LICENSE** file for the complete license text.