README.md

---
license: apache-2.0
license_link: https://www.apache.org/licenses/LICENSE-2.0

language:
- en

base_model:
- Qwen/Qwen3-4B

datasets:
- HuggingFaceTB/smoltalk
- agentica-org/DeepCoder-Preview-Dataset

pipeline_tag: text-generation
library_name: transformers

tags:
- qwen3
- computer-science
- software-engineering
- programming
- python
- code-generation
- debugging
- transformers
- pytorch
---

# Qwen3-4B-Computer-Science

Qwen3-4B-Computer-Science is a supervised fine-tuned language model based on **Qwen/Qwen3-4B**, designed for computer science and software engineering tasks.

This repository contains the merged BF16 checkpoint compatible with the Hugging Face Transformers ecosystem.

---

# Model Summary

The model specializes in programming-oriented instruction following across multiple computer science domains, including software engineering, debugging, algorithms, testing, and technical reasoning.

Training was performed using parameter-efficient supervised fine-tuning (LoRA). The released checkpoint contains merged BF16 weights and can be used directly without PEFT adapters.

---

# Motivation

General-purpose language models provide strong performance across many domains but are not specifically optimized for computer science workflows.

Qwen3-4B-Computer-Science aims to improve programming-oriented instruction following while preserving the capabilities of the original Qwen3-4B base model.

---

# Model Details

| Field | Value |
|------|------|
| Model Name | Qwen3-4B-Computer-Science |
| Base Model | [Qwen/Qwen3-4B](https://huggingface.co/Qwen/Qwen3-4B) |
| Model Type | Causal Language Model |
| Architecture | Decoder-only Transformer |
| Parameters | 4 Billion |
| Fine-Tuning | Supervised Fine-Tuning (SFT) |
| Fine-Tuning Method | LoRA |
| Training Strategy | Distributed Data Parallel (DDP) |
| Released Weights | Merged BF16 |
| Framework | Hugging Face Transformers |
| Primary Language | English |

---

# Training

Training was performed using supervised fine-tuning (SFT) with parameter-efficient fine-tuning (LoRA).

Optimization utilized Distributed Data Parallel (DDP). After training, the LoRA adapters were merged into the base model to produce the released BF16 checkpoint.

The published model does not require PEFT adapters during inference.

---

# Training Data

The final training corpus contains **60,989** training examples and **512** evaluation examples.

| Dataset | Configuration | License | Train | Eval |
|---------|--------------|---------|------:|-----:|
| HuggingFaceTB/smoltalk | smol-magpie-ultra | Apache-2.0 | 49,584 | 416 |
| agentica-org/DeepCoder-Preview-Dataset | primeintellect | MIT | 11,405 | 96 |

---

# Dataset Attribution

The model was fine-tuned using publicly available datasets released under their respective licenses.

| Dataset | Configuration | License |
|---------|--------------|---------|
| HuggingFaceTB/smoltalk | smol-magpie-ultra | Apache-2.0 |
| agentica-org/DeepCoder-Preview-Dataset | primeintellect | MIT |

Credit for the datasets belongs to their respective authors.

---

# Intended Use

Recommended applications include:

- Software engineering
- Programming assistance
- Python development
- Code generation
- Code explanation
- Debugging
- Unit testing
- Technical documentation
- Computer science education

---

# Capabilities

The model has been fine-tuned for:

- Programming-oriented instruction following
- Code generation
- Code completion
- Code explanation
- Refactoring
- Debugging
- Algorithm implementation
- Standard library usage
- Technical reasoning

The model inherits the general instruction-following capabilities of Qwen3-4B.

---

# Installation

```bash
pip install -U transformers accelerate torch
```

---

# Usage

```python
from transformers import AutoTokenizer
from transformers import AutoModelForCausalLM

model_name = "Irfanuruchi/Qwen3-4B-Computer-Science"

tokenizer = AutoTokenizer.from_pretrained(model_name)

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto",
)
```

## Example

```python
messages = [
    {
        "role": "user",
        "content": "Implement binary search in Python."
    }
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)

inputs = tokenizer(text, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=512,
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

---

# Hardware Requirements

This repository contains merged BF16 weights.

Memory requirements depend on the selected precision and inference backend.

Users with limited GPU memory are encouraged to use the GGUF release when available.

---

# Limitations

Although specialized for computer science tasks, the model remains a probabilistic language model.

Outputs should be reviewed before use in production environments.

The model may:

- generate incorrect code
- hallucinate APIs or libraries
- produce incomplete implementations
- misunderstand project-specific context

---

# License

This repository is released under the **Apache License 2.0**.

## Base Model

This project is derived from **Qwen/Qwen3-4B**, which is distributed under the Apache License 2.0.

## Training Data

The datasets retain their original licenses.

| Dataset | License |
|---------|---------|
| HuggingFaceTB/smoltalk | Apache-2.0 |
| agentica-org/DeepCoder-Preview-Dataset | MIT |

---

# Acknowledgements

This project builds upon the work of:

- Alibaba Qwen Team
- Hugging Face
- HuggingFaceTB
- Agentica
- Unsloth

The contributions of these open-source projects made this work possible.

---

# Citation

```bibtex
@misc{uruci2026qwen3cs,
  title={Qwen3-4B-Computer-Science},
  author={Irfan Uruçi},
  year={2026},
  publisher={Hugging Face},
  howpublished={https://huggingface.co/Irfanuruchi/Qwen3-4B-Computer-Science}
}
```

---

# Contact

Questions, bug reports, and suggestions are welcome through the Hugging Face repository discussions.