README.md

---
language:
- en
license: apache-2.0
base_model: Irfanuruchi/Qwen3-4B-Computer-Science
library_name: optimum
pipeline_tag: text-generation
tags:
- qwen3
- computer-science
- software-engineering
- coding
- openvino
- int4
- cpu
- optimization
---

# Qwen3-4B-Computer-Science OpenVINO INT4

This repository provides an **OpenVINO INT4** version of **Qwen3-4B-Computer-Science**, optimized for efficient inference on Intel CPUs and other OpenVINO-supported hardware.

The model has been exported using **Optimum Intel** with OpenVINO IR format and INT4 weight compression, providing a significantly smaller footprint while maintaining strong performance for software engineering and computer science workloads.

---

# Model Information

| Property | Value |
|----------|-------|
| Base Model | Irfanuruchi/Qwen3-4B-Computer-Science |
| Architecture | Qwen3 |
| Parameters | ~4 Billion |
| Format | OpenVINO IR |
| Weight Compression | INT4 Asymmetric |
| Group Size | 128 |
| Framework | OpenVINO + Optimum Intel |
| Primary Device | CPU |
| License | Apache License 2.0 |

---

# Features

- OpenVINO IR optimized for CPU inference
- INT4 asymmetric weight compression
- Reduced storage requirements
- Lower memory usage
- Fast local inference
- Compatible with Hugging Face Transformers through Optimum Intel
- Includes tokenizer, detokenizer, and chat template

---

# Export Configuration

The model was exported using:

```bash
optimum-cli export openvino \
    --model Irfanuruchi/Qwen3-4B-Computer-Science \
    --task text-generation-with-past \
    --weight-format int4 \
    Qwen3-4B-Computer-Science-OpenVINO-INT4
```

Compression statistics:

- **252** transformer layers compressed to **INT4 asymmetric** (group size 128)
- **1** auxiliary layer stored as **INT8 per-channel**

---

# Installation

```bash
pip install -U openvino optimum-intel transformers
```

---

# Example Usage

```python
from transformers import AutoTokenizer
from optimum.intel.openvino import OVModelForCausalLM

model_id = "Irfanuruchi/Qwen3-4B-Computer-Science-OpenVINO-INT4"

tokenizer = AutoTokenizer.from_pretrained(model_id)

model = OVModelForCausalLM.from_pretrained(
    model_id,
    device="CPU",
)

messages = [
    {
        "role": "system",
        "content": "You are a computer science assistant."
    },
    {
        "role": "user",
        "content": "Explain Floyd's cycle detection algorithm."
    },
]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)

inputs = tokenizer(prompt, return_tensors="pt")

outputs = model.generate(
    **inputs,
    max_new_tokens=256,
    do_sample=False,
)

response = tokenizer.decode(
    outputs[0][inputs["input_ids"].shape[1]:],
    skip_special_tokens=True,
)

print(response)
```

---

# Validation

The exported model has been successfully validated using:

- OpenVINO 2026.2.1
- CPU inference
- Chat template support
- Greedy decoding
- Software engineering benchmark prompts

Validation confirmed successful generation of technically correct programming responses, including algorithm implementation and complexity analysis.

---

# Intended Use

This model is intended for:

- Software engineering assistance
- Computer science education
- Code generation
- Code review
- Debugging
- Algorithm design
- Technical documentation
- Programming question answering

---

# Limitations

As with other large language models, outputs should be reviewed before production use. The model may occasionally:

- Generate incorrect or incomplete code
- Hallucinate APIs or implementation details
- Produce inefficient solutions
- Misinterpret ambiguous instructions

INT4 compression may also introduce minor differences compared to higher-precision variants.

---

# License

This model is distributed under the **Apache License 2.0**.

Please refer to the included **LICENSE** file for the complete license text and attribution requirements.