Instructions to use Irfanuruchi/Qwen3-4B-Computer-Science-Models with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Irfanuruchi/Qwen3-4B-Computer-Science-Models with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Irfanuruchi/Qwen3-4B-Computer-Science-Models")

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("Irfanuruchi/Qwen3-4B-Computer-Science-Models", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Irfanuruchi/Qwen3-4B-Computer-Science-Models with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Irfanuruchi/Qwen3-4B-Computer-Science-Models"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Irfanuruchi/Qwen3-4B-Computer-Science-Models",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/Irfanuruchi/Qwen3-4B-Computer-Science-Models

SGLang

How to use Irfanuruchi/Qwen3-4B-Computer-Science-Models with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Irfanuruchi/Qwen3-4B-Computer-Science-Models" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Irfanuruchi/Qwen3-4B-Computer-Science-Models",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Irfanuruchi/Qwen3-4B-Computer-Science-Models" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Irfanuruchi/Qwen3-4B-Computer-Science-Models",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use Irfanuruchi/Qwen3-4B-Computer-Science-Models with Docker Model Runner:
```
docker model run hf.co/Irfanuruchi/Qwen3-4B-Computer-Science-Models
```

Irfanuruchi commited on 6 days ago

Commit

4ad30f9

verified ·

1 Parent(s): 892dba9

Update README.md

Browse files

Files changed (1) hide show

README.md +34 -16

README.md CHANGED Viewed

@@ -9,7 +9,6 @@ pipeline_tag: text-generation
 inference: false
 tags:
   - qwen3
-  - qwen3-for-causal-lm
   - computer-science
   - software-engineering
   - programming
@@ -18,15 +17,17 @@ tags:
   - optimum-intel
   - int4
   - quantized
-  - cpu
   - transformers
 ---
 # Qwen3-4B-Computer-Science OpenVINO INT4
-This repository provides an **OpenVINO INT4** version of **Qwen3-4B-Computer-Science**, optimized for efficient inference using **Intel OpenVINO** and **Optimum Intel**.
-The model is optimized for software engineering and computer science workloads while significantly reducing storage requirements and memory usage through INT4 weight compression, enabling efficient CPU inference on OpenVINO-supported hardware.
 ---
@@ -37,24 +38,26 @@ The model is optimized for software engineering and computer science workloads w
 | Base Model | Irfanuruchi/Qwen3-4B-Computer-Science |
 | Base Architecture | Qwen3 |
 | Parameters | ~4 Billion |
-| Model Format | OpenVINO IR |
 | Weight Compression | INT4 Asymmetric |
 | Compression Group Size | 128 |
-| Runtime | OpenVINO Runtime |
 | Library | Optimum Intel |
-| Primary Hardware | CPU |
 | License | Apache License 2.0 |
 ---
 # Features
-- Optimized OpenVINO IR format
 - INT4 asymmetric weight compression
 - Reduced storage footprint
 - Lower memory usage
-- CPU-optimized inference
-- Compatible with Hugging Face Transformers through Optimum Intel
 - Exported tokenizer and detokenizer
 - Chat template included
 - Ready for local deployment
@@ -142,8 +145,8 @@ print(response)
 The exported model was successfully validated using:
-- OpenVINO 2026.2.1
-- CPU inference
 - Chat template support
 - Greedy decoding
 - Software engineering benchmark prompts
@@ -182,7 +185,20 @@ The **Qwen3-4B-Computer-Science** release family currently includes:
 | MLX BF16 | ✅ |
 | OpenVINO INT4 | ✅ |
-Each release is maintained in its own repository with runtime-specific documentation, usage examples, integrity verification files, and configuration for its respective inference backend.
 ---
@@ -195,9 +211,9 @@ Like other large language models, this model may occasionally:
 - Produce inefficient implementations
 - Misinterpret ambiguous instructions
-All generated code should be reviewed and tested before use in production or safety-critical environments.
-INT4 weight compression may introduce minor quality differences compared with higher-precision model variants.
 ---
@@ -205,4 +221,6 @@ INT4 weight compression may introduce minor quality differences compared with hi
 This repository is distributed under the **Apache License 2.0**.
-See the included **LICENSE** file for the complete license text and applicable attribution requirements.

 inference: false
 tags:
   - qwen3
   - computer-science
   - software-engineering
   - programming
   - optimum-intel
   - int4
   - quantized
   - transformers
+  - cpu
+  - gpu
+  - npu
 ---
 # Qwen3-4B-Computer-Science OpenVINO INT4
+This repository provides an **OpenVINO INT4** deployment of **Qwen3-4B-Computer-Science**, optimized for efficient inference using **Intel® OpenVINO™** and **Optimum Intel**.
+The model is designed for software engineering and computer science workloads while significantly reducing storage requirements and memory usage through INT4 weight compression. It enables efficient deployment across OpenVINO-supported hardware while preserving the capabilities of the original model.
 ---
 | Base Model | Irfanuruchi/Qwen3-4B-Computer-Science |
 | Base Architecture | Qwen3 |
 | Parameters | ~4 Billion |
+| Deployment Format | OpenVINO IR |
 | Weight Compression | INT4 Asymmetric |
 | Compression Group Size | 128 |
+| Runtime | Intel® OpenVINO™ Runtime |
 | Library | Optimum Intel |
+| Supported Hardware | OpenVINO-supported devices* |
 | License | Apache License 2.0 |
+> **\*** Supported execution devices depend on the installed OpenVINO Runtime, operating system, drivers, and available hardware. Depending on the platform, inference may be executed on supported CPUs, integrated GPUs, NPUs, or other OpenVINO-compatible accelerators.
 ---
 # Features
+- OpenVINO IR deployment format
 - INT4 asymmetric weight compression
 - Reduced storage footprint
 - Lower memory usage
+- Efficient inference on OpenVINO-supported hardware
+- Compatible with Optimum Intel and Hugging Face Transformers
 - Exported tokenizer and detokenizer
 - Chat template included
 - Ready for local deployment
 The exported model was successfully validated using:
+- Intel® OpenVINO™ Runtime 2026.2.1
+- Local OpenVINO inference
 - Chat template support
 - Greedy decoding
 - Software engineering benchmark prompts
 | MLX BF16 | ✅ |
 | OpenVINO INT4 | ✅ |
+Each release is maintained in its own repository with runtime-specific documentation, usage examples, integrity verification files, and configuration tailored to its target inference backend.
+---
+# Credits
+This release builds upon the work of several open-source projects and communities:
+- **Qwen Team** for the Qwen3 foundation model architecture.
+- **Intel® OpenVINO™ Toolkit** for the OpenVINO Runtime and deployment framework.
+- **Hugging Face** for the Transformers ecosystem and the Optimum Intel integration.
+- **Optimum Intel** for OpenVINO model export and runtime integration.
+This repository provides an **OpenVINO INT4 deployment** of the original **Qwen3-4B-Computer-Science** model for efficient inference on OpenVINO-supported hardware.
 ---
 - Produce inefficient implementations
 - Misinterpret ambiguous instructions
+INT4 weight compression may introduce minor quality differences compared to higher-precision variants.
+All generated code should be reviewed and tested before use in production or safety-critical environments.
 ---
 This repository is distributed under the **Apache License 2.0**.
+This release is an OpenVINO INT4 conversion of the original **Qwen3-4B-Computer-Science** model and retains the licensing and attribution requirements applicable to the original project.
+See the included **LICENSE** file for the complete license text.